The new source tag_pool.c manages a pool of reference counted tag_item
objects. This is used to merge tag items of the same type and value,
saving lots of memory. Formerly, only the value itself was pooled,
wasting memory for all the pointers and tag_item structs.
The following results were measured with massif. Started MPD on
amd64, typed "mpc", no song being played. My music database contains
35k tagged songs. The results are what massif reports as "peak".
0.13.2: total 14,131,392; useful 11,408,972; extra 2,722,420
eric: total 18,370,696; useful 15,648,182; extra 2,722,514
mk f34f694: total 15,833,952; useful 13,111,470; extra 2,722,482
mk now: total 12,837,632; useful 10,626,383; extra 2,211,249
This patch set saves 20% memory, and does a good job in reducing heap
fragmentation.
This patch makes MPD consume much more memory because string pooling
is disabled, but it prepares the next bunch of patches. Replace the
code in tagTracker.c with naive algorithms without the tree code. For
now, this should do; later we should find better algorithms,
especially for getNumberOfTagItems(), which has become wasteful with
temporary memory.
I don't believe "interface" is a good name for something like
"connection by a client to MPD", let's call it "client". This is the
first patch in the series which changes the name, beginning with the
file name.
linux/list.h is a nice doubly linked list library - it is lightweight
and powerful at the same time. It will be useful later, when we begin
to allocate client structures dynamically. Import it, and strip out
all the stuff which we are not going to use.
It should be obvious in which thread or context a function is being
executed at runtime. The code which was left in decode.c is for the
decoder thread itself; give the file a better name.
decoder_initialized() sets the state to DECODE_STATE_DECODE and wakes
up the player thread. It is called by the decoder plugin after its
internal initialization is finished. More arguments will be added
later to prevent direct accesses to the DecoderControl struct.
The decoder struct should later be made opaque to the decoder plugin,
because maintaining a stable struct ABI is quite difficult. The ABI
should only consist of a small number of stable functions.
The auth code also has some ugly usages of string generation
which I will eventually replace with something nicer...
git-svn-id: https://svn.musicpd.org/mpd/trunk@7387 09075e82-0dd4-0310-85a5-a0d7c8717e4f
This piece of code is from the JACK Audio Connection Kit
(trimmed down a bit for better readability).
The vector functions now reuse the common iovec struct used by
writev/readv instead of reinventing an identical but
differently-named struct.
From the comments:
> ISO/POSIX C version of Paul Davis's lock free ringbuffer C++ code.
> This is safe for the case of one read thread and one write thread.
License is LGPL 2.1 or later
git-svn-id: https://svn.musicpd.org/mpd/trunk@7386 09075e82-0dd4-0310-85a5-a0d7c8717e4f
The select() in the main event loop blocks now (saving us many
unnecessary wakeups). This interacted badly with the threads
that were trying to wakeup the main task via
pthread_cond_signal() since the main task was not blocked
on a condition variable, but on select().
So now if we detect a need to wakeup the player, we write
to a pipe which select() is watching instead of blindly
calling pthread_cond_signal().
git-svn-id: https://svn.musicpd.org/mpd/trunk@7347 09075e82-0dd4-0310-85a5-a0d7c8717e4f
It is way more complicated than it should be; and
locking it for thread-safety is too difficult.
[merged r7183 from branches/ew]
git-svn-id: https://svn.musicpd.org/mpd/trunk@7241 09075e82-0dd4-0310-85a5-a0d7c8717e4f
When the decoder process is faster than the player process, all
decodedd buffers are full at some point in time. The decoder has to
wait for buffers to become free (finished playing). It used to do
this by polling the buffer status 100 times a second.
This generates a lot of unnecessary CPU wakeups. This patch adds a
way for the player process to notify the decoder process that it may
continue its work.
We could use pthread_cond for that, unfortunately inter-process
mutexes/conds are not supported by some kernels (Linux), so we cannot
use this light-weight method until mpd moves to using threads instead
of processes. The other method would be semaphores, which
historically are global resources with a unique name; this historic
API is cumbersome, and I wanted to avoid it.
I came up with a quite naive solution for now: I create an anonymous
pipe with pipe(), and the decoder process reads on that pipe. Until
the player process sends data on it as a signal, the decoder process
blocks.
This can be optimized in a number of ways:
- if the decoder process is still working (instead of waiting for
buffers), we could save the write() system call, since there is
nobody waiting for the notification.
[ew: I tried this using a counter in shared memory, didn't help]
- the pipe buffer will be full at some point, when the decoder thread
is too slow. For this reason, the writer side of the pipe is
non-blocking, and mpd can ignore the resulting EWOULDBLOCK.
- since we have shared memory, we could check whether somebody is
actually waiting without a context switch, and we could just not
write the notification byte.
[ew: tried same method/result as first point above]
- if there is already a notification in the pipe, we could also not
write another one.
[ew: tried same method/result as first/third points above]
- the decoder will only consume 64 bytes at a time. If the pipe
buffer is full, this will result in a lot of read() invocations.
This does not hurt badly, but on a heavily loaded system, this might
add a little bit more load. The preceding optimizations however
are able eliminate the this.
- finally, we should use another method for inter process
notifications - maybe kill() or just make mpd use threads, finally.
In spite of all these possibilities to optimize this code further,
this pipe notification trick is faster than the 100 Hz poll. On my
machine, it reduced the number of wakeups to less than 30%.
git-svn-id: https://svn.musicpd.org/mpd/trunk@7215 09075e82-0dd4-0310-85a5-a0d7c8717e4f
This reduces the text size of the binary slightly when zeroconf
support is not built, and keeps the interface code cleaner as
well.
git-svn-id: https://svn.musicpd.org/mpd/trunk@7133 09075e82-0dd4-0310-85a5-a0d7c8717e4f
thread-safety work in preparation for rewrite to use pthreads
Expect no regressions against trunk (r7078), possibly minor
performance improvements in update (due to fewer heap
allocations), but increased stack usage.
Applied the following patches:
* maxpath_str for reentrancy (temporary fix, reverted)
* path: start working on thread-safe variants of these methods
* Re-entrancy work on path/character-set conversions
* directory.c: exploreDirectory() use reentrant functions here
* directory/update: more use of reentrant functions + cleanups
* string_toupper: a strdup-less version of strDupToUpper
* get_song_url: a static-variable-free version of getSongUrl()
* Use reentrant/thread-safe get_song_url everywhere
* replace rmp2amp with the reentrant version, rmp2amp_r
* Get rid of the non-reentrant/non-thread-safe rpp2app, too.
* buffer2array: assert strdup() returns a usable value in unit tests
* replace utf8ToFsCharset and fsCharsetToUtf8 with thread-safe variants
* fix storing playlists w/o absolute paths
* parent_path(), a reentrant version of parentPath()
* parentPath => parent_path for reentrancy and thread-safety
* allow "make test" to automatically run embedded unit tests
* remove convStrDup() and maxpath_str()
* use MPD_PATH_MAX everywhere instead of MAXPATHLEN
* path: get rid of appendSlash, pfx_path and just use pfx_dir
* get_song_url: fix the ability to play songs in the top-level music_directory
git-svn-id: https://svn.musicpd.org/mpd/trunk@7106 09075e82-0dd4-0310-85a5-a0d7c8717e4f
implementation, and fixing it is a big enough job that I don't know when
I'll get around to it. Probably best just starting from scratch anyhow.
git-svn-id: https://svn.musicpd.org/mpd/trunk@5373 09075e82-0dd4-0310-85a5-a0d7c8717e4f
This patch massively reduces the amount of heap allocations at
the interface/command layer. Most commands with minimal output
should not allocate memory from the heap at all. Things like
repeatedly polling status, currentsong, and volume changes
should be faster as a result, and more importantly, not a source
of memory fragmentation.
These changes should be safe in that there's no way for a
remote-client to corrupt memory or otherwise do bad stuff to
MPD, but an extra set of eyes to review would be good. Of
course there's never any warranty :)
No longer do we use FILE * structures in the interface, which means
we don't have to allocate any new memory for most connections.
Now, before you go on about losing the buffering that FILE *
+implies+, remember that myfprintf() never took advantage of
any of the stdio buffering features.
To reduce the diff and make bugs easier to spot in the diff,
I've kept myfprintf in places where we write to files (and not
network interfaces). Expect myfprintf to go away entirely soon
(we'll use fprintf for writing regular files).
git-svn-id: https://svn.musicpd.org/mpd/trunk@4483 09075e82-0dd4-0310-85a5-a0d7c8717e4f
sparse is a semantic parser developed for the Linux kernel,
but works for any project written (ANSI) C.
You can get sparse via git here:
git clone git://www.kernel.org/pub/scm/devel/sparse/sparse.git
git-svn-id: https://svn.musicpd.org/mpd/trunk@4377 09075e82-0dd4-0310-85a5-a0d7c8717e4f
