Taming the directory.c monster, part II: move the database management
stuff to database. directory.c should only contain code which works
on directory objects.
The source directory.c mixes several libraries: directory object
management, database management and database update, resulting in a
1000+ line monster. Move the whole database update code to update.c.
Having all functions as static (non-inline) functions generates GCC
warnings, and duplicates binary code across several object files.
Most of dirvec's methods are too complex for becoming inline
functions. Move them all to dirvec.c and publish the prototypes in
dirvec.h.
* Add missing headers in Makefile.am
* remove mp4ff.dsp (Win32 crap)
* Add scripts, m4, bs, autogen.sh to allow for hotfixes by the
SCM-challenged. (downloading the source via git is NOT a
lightweight operation for everybody).
Since we use a C99 compiler now, we can assert that the C99 standard
headers are available, no need for complicated compile time checks.
Kill mpd_types.h.
We have eliminated direct accesses to the audio_output struct from
the all output plugins. Make it opaque for them, and move its real
declaration to output_internal.h, similar to decoder_internal.h.
Pass the opaque structure to plugin.init() only, which will return the
plugin's data pointer on success, and NULL on failure. This data
pointer will be passed to all other methods instead of the
audio_output struct.
Reduce direct accesses to the audio_output struct from the plugins:
this time, eliminate all accesses to audio_output.name. The name is
required by some plugins for log messages.
To keep I/O nastiness and latencies away from the core, move the audio
output code to a separate thread, one per output. The thread is
created on demand, and currently runs until mpd exits.
Our linked-list implementation is wasteful and the
SongList isn't modified enough to benefit from being a linked
list. So use a more compact array of song pointers which
saves ~200K on a library with ~9K songs (on x86-32).
[mk: moved this patch after "Refactor and cleanup of shout Ogg and MP3
audio outputs". The original commit message follows, although it is
outdated:]
Creation of shout_mp3 audio output plugin. Basically I just copied the
existing shout plugin and replaced ogg with lame. Uses lame for mp3
encoding. Next step is to pull common functionality out of each shout
plugin and share it between them.
Configuration options for "shout_mp3" are the same as for "shout".
Begin dividing audioOutput_shout.c: move everything OGG Vorbis related
to audioOutput_shout_ogg.c. The header audioOutput_shout.h has to
keep its dependency on vorbis/vorbisenc.h, because it needs the vorbis
encoder types.
For this patch, we have to export several internal functions with
generic names to the ABI; these will be removed later when the encoder
plugin patches are merged.
With a large music database, the linear string collection in
tagTracker.c becomes very slow. We implemented that in a
quick'n'dirty fashion when we removed tree.c, and now we rewrite it
using the fast hashed string set.
"struct strset" is a hashed string set: you can add strings to this
library, and it stores them as a set of unique strings. You can get
the size of the set, and you can enumerate through all values.
This will be used to replace the linear tagTracker library.
Instead of having to register each output plugin, store them
statically in an array. This eliminates the need for the List library
here, and saves some small allocations during startup.
All callers of fdprintf() have been converted to client_printf() or
fprintf(); it is time to remove this clumsy hack now. We can also
remove client_print() which took a file descriptor as parameter.
Currently, when the tag cache is being serialized to hard disk, the
stdio buffer is flushed before every song, because tag_print.c
performs unbuffered writes on the raw file descriptor. Unfortunately,
the fdprintf() API allows buffered I/O only for a client connection by
looking up the client pointer owning the file descriptor - for stdio,
this is not possible. To re-enable proper stdio buffering, we have to
duplicate the tag_print.c code without fprintf() instead of our custom
fdprintf() hack. Add this duplicated code to tag_save.c.
Move everything which dumps song information (via tag_print.c) to a
separate source file. song_print.c gets code which writes song data
to the client; song_save.c is responsible for serializing songs from
the tag cache.
The new source tag_pool.c manages a pool of reference counted tag_item
objects. This is used to merge tag items of the same type and value,
saving lots of memory. Formerly, only the value itself was pooled,
wasting memory for all the pointers and tag_item structs.
The following results were measured with massif. Started MPD on
amd64, typed "mpc", no song being played. My music database contains
35k tagged songs. The results are what massif reports as "peak".
0.13.2: total 14,131,392; useful 11,408,972; extra 2,722,420
eric: total 18,370,696; useful 15,648,182; extra 2,722,514
mk f34f694: total 15,833,952; useful 13,111,470; extra 2,722,482
mk now: total 12,837,632; useful 10,626,383; extra 2,211,249
This patch set saves 20% memory, and does a good job in reducing heap
fragmentation.
This patch makes MPD consume much more memory because string pooling
is disabled, but it prepares the next bunch of patches. Replace the
code in tagTracker.c with naive algorithms without the tree code. For
now, this should do; later we should find better algorithms,
especially for getNumberOfTagItems(), which has become wasteful with
temporary memory.
I don't believe "interface" is a good name for something like
"connection by a client to MPD", let's call it "client". This is the
first patch in the series which changes the name, beginning with the
file name.
linux/list.h is a nice doubly linked list library - it is lightweight
and powerful at the same time. It will be useful later, when we begin
to allocate client structures dynamically. Import it, and strip out
all the stuff which we are not going to use.
It should be obvious in which thread or context a function is being
executed at runtime. The code which was left in decode.c is for the
decoder thread itself; give the file a better name.
decoder_initialized() sets the state to DECODE_STATE_DECODE and wakes
up the player thread. It is called by the decoder plugin after its
internal initialization is finished. More arguments will be added
later to prevent direct accesses to the DecoderControl struct.
The decoder struct should later be made opaque to the decoder plugin,
because maintaining a stable struct ABI is quite difficult. The ABI
should only consist of a small number of stable functions.
The auth code also has some ugly usages of string generation
which I will eventually replace with something nicer...
git-svn-id: https://svn.musicpd.org/mpd/trunk@7387 09075e82-0dd4-0310-85a5-a0d7c8717e4f
This piece of code is from the JACK Audio Connection Kit
(trimmed down a bit for better readability).
The vector functions now reuse the common iovec struct used by
writev/readv instead of reinventing an identical but
differently-named struct.
From the comments:
> ISO/POSIX C version of Paul Davis's lock free ringbuffer C++ code.
> This is safe for the case of one read thread and one write thread.
License is LGPL 2.1 or later
git-svn-id: https://svn.musicpd.org/mpd/trunk@7386 09075e82-0dd4-0310-85a5-a0d7c8717e4f
The select() in the main event loop blocks now (saving us many
unnecessary wakeups). This interacted badly with the threads
that were trying to wakeup the main task via
pthread_cond_signal() since the main task was not blocked
on a condition variable, but on select().
So now if we detect a need to wakeup the player, we write
to a pipe which select() is watching instead of blindly
calling pthread_cond_signal().
git-svn-id: https://svn.musicpd.org/mpd/trunk@7347 09075e82-0dd4-0310-85a5-a0d7c8717e4f
It is way more complicated than it should be; and
locking it for thread-safety is too difficult.
[merged r7183 from branches/ew]
git-svn-id: https://svn.musicpd.org/mpd/trunk@7241 09075e82-0dd4-0310-85a5-a0d7c8717e4f
When the decoder process is faster than the player process, all
decodedd buffers are full at some point in time. The decoder has to
wait for buffers to become free (finished playing). It used to do
this by polling the buffer status 100 times a second.
This generates a lot of unnecessary CPU wakeups. This patch adds a
way for the player process to notify the decoder process that it may
continue its work.
We could use pthread_cond for that, unfortunately inter-process
mutexes/conds are not supported by some kernels (Linux), so we cannot
use this light-weight method until mpd moves to using threads instead
of processes. The other method would be semaphores, which
historically are global resources with a unique name; this historic
API is cumbersome, and I wanted to avoid it.
I came up with a quite naive solution for now: I create an anonymous
pipe with pipe(), and the decoder process reads on that pipe. Until
the player process sends data on it as a signal, the decoder process
blocks.
This can be optimized in a number of ways:
- if the decoder process is still working (instead of waiting for
buffers), we could save the write() system call, since there is
nobody waiting for the notification.
[ew: I tried this using a counter in shared memory, didn't help]
- the pipe buffer will be full at some point, when the decoder thread
is too slow. For this reason, the writer side of the pipe is
non-blocking, and mpd can ignore the resulting EWOULDBLOCK.
- since we have shared memory, we could check whether somebody is
actually waiting without a context switch, and we could just not
write the notification byte.
[ew: tried same method/result as first point above]
- if there is already a notification in the pipe, we could also not
write another one.
[ew: tried same method/result as first/third points above]
- the decoder will only consume 64 bytes at a time. If the pipe
buffer is full, this will result in a lot of read() invocations.
This does not hurt badly, but on a heavily loaded system, this might
add a little bit more load. The preceding optimizations however
are able eliminate the this.
- finally, we should use another method for inter process
notifications - maybe kill() or just make mpd use threads, finally.
In spite of all these possibilities to optimize this code further,
this pipe notification trick is faster than the 100 Hz poll. On my
machine, it reduced the number of wakeups to less than 30%.
git-svn-id: https://svn.musicpd.org/mpd/trunk@7215 09075e82-0dd4-0310-85a5-a0d7c8717e4f
This reduces the text size of the binary slightly when zeroconf
support is not built, and keeps the interface code cleaner as
well.
git-svn-id: https://svn.musicpd.org/mpd/trunk@7133 09075e82-0dd4-0310-85a5-a0d7c8717e4f
thread-safety work in preparation for rewrite to use pthreads
Expect no regressions against trunk (r7078), possibly minor
performance improvements in update (due to fewer heap
allocations), but increased stack usage.
Applied the following patches:
* maxpath_str for reentrancy (temporary fix, reverted)
* path: start working on thread-safe variants of these methods
* Re-entrancy work on path/character-set conversions
* directory.c: exploreDirectory() use reentrant functions here
* directory/update: more use of reentrant functions + cleanups
* string_toupper: a strdup-less version of strDupToUpper
* get_song_url: a static-variable-free version of getSongUrl()
* Use reentrant/thread-safe get_song_url everywhere
* replace rmp2amp with the reentrant version, rmp2amp_r
* Get rid of the non-reentrant/non-thread-safe rpp2app, too.
* buffer2array: assert strdup() returns a usable value in unit tests
* replace utf8ToFsCharset and fsCharsetToUtf8 with thread-safe variants
* fix storing playlists w/o absolute paths
* parent_path(), a reentrant version of parentPath()
* parentPath => parent_path for reentrancy and thread-safety
* allow "make test" to automatically run embedded unit tests
* remove convStrDup() and maxpath_str()
* use MPD_PATH_MAX everywhere instead of MAXPATHLEN
* path: get rid of appendSlash, pfx_path and just use pfx_dir
* get_song_url: fix the ability to play songs in the top-level music_directory
git-svn-id: https://svn.musicpd.org/mpd/trunk@7106 09075e82-0dd4-0310-85a5-a0d7c8717e4f
