HTTP servers respond with "416 Requested Range Not Satisfiable" when a
client attempts to seek to the end of the file. Catch this special
case in input_curl_seek(). This fixes a glitch in the ogg vorbis
decoder plugin.
Since we are using curl_multi_info_read() / CURLMSG_DONE for detecting
end-of-response, we can remove all running_handles==0 checks. For
some reason, that has never worked correctly.
curl_multi_info_read() is the authoritative source of the
"end-of-response" information. Always set c->eof when a CURLMSG_DONE
message is received, and check the result (success/failure) after
that.
When a global audio format is configured (setting
"audio_output_format"), decoder_data() overwrote the "length"
parameter with the size of the output buffer (result of
pcm_convert_size()). Declare a separate variable for the output
buffer length.
neaacdec.h declares all arguments as "unsigned long", but internally
expects uint32_t pointers. This triggers gcc warnings on 64 bit
architectures. To avoid that, make configure.ac detect whether we're
using Debian's corrected headers or the original libfaad headers. In
any case, pass a pointer to an uint32_t, conditionally casted to
"unsigned long*".
alloca() is not a portable function. Don't use it. Using
strncasecmp() is much more efficient anyway, because no memory needs
to be allocated and copied.
Don't send a "next song" request to the main thread when the current
song hasn't started playing yet, i.e. there are already two different
songs in the music pipe. This would erase information about the song
boundary within the music pipe, and thus triggered an assertion
failure. The bug could occur when playing very short songs which fit
into the pipe as a whole.
Fix a deadlock: when the decoder waited for buffer space, the player
could enter a deadlock situation because it waits for more chunks for
crossfading chunks. Signal the decoder before entering notify_wait().
The wavpack open function gives us an option called OPEN_STREAMING. This
provides more robust and error tolerant playback, but it automatically
disables seeking. (More exactly the wavpack lib will not return the
length information.) So, if the stream is already not seekable we can
use this option safely.
Seeking was somewhat broken in some decoder plugins because they sent
empty chunks, and never got a command. Check the decoder command
before doing anything else in decoder_data().
According to the documentation, mpc_decoder_decode() returns an
mpc_uint32_t. Since the special return value (mpc_uint32_t)-1
translates to a very large long integer, this may cause segmentation
faults if not interpreted properly.
Don't split the buffer conversion loop. When libmpcdec returns a
chunk, convert and send the whole chunk at a time. This moves several
checks out of the loop, and greatly improves performance.
Parse ID3 tags, even when they are in the middle of the stream. Very
few streams provide embedded ID3 tags. Most of them send only
Shoutcast "icy" tags, which limits the practical usefulness of this
patch.
When a command is received, decode_next_frame_header() and
decodeNextFrame() return DECODE_BREAK. This is already checked by
both callers, which means that we can eliminate lots of
decoder_get_command() checks.
When a tag is updated, the old tag was freed before the new one was
created. Reverse the order to be sure that other threads always see a
valid pointer.
This still leaves a possible race condition, but it will be addressed
later.
The stream_decode() and file_decode() methods returned a boolean,
indicating whether they were able to decode the song. This is
redundant, since we already know that: if decoder_initialized() has
been called (and dc.state==DECODE), the plugin succeeded. Change both
methods to return void.
The currently replay_gain_apply() implementation duplicates code from
pcm_volume(), except that it uses a floating point scale. Eliminate
all duplicated code from and make it utilize the pcm_volume() library
function. This introduces replay gain support for 24 bit audio.
It may be desirable to change the range of integer volume levels
(e.g. to 1024, which may utilize shifts instead of expensive integer
divisions). Introduce the constant PCM_VOLUME_1 which describes the
integer value for "100% volume". This is currently 1000.
The function simplifies wavpack_replaygain(), because it already
contains the float parser, and it works with a fixed buffer instead of
doing expensive heap allocations.
The assertion on dc.state in decoder_read() was too strict: when a
decoder tried to call decoder_read() from tag_dup(), the decoder state
was NONE. Allow this special case.
The flac plugin wasn't initialized properly when an OGG file was being
decoded. For some reason, flac_process_metadata() was explicitly not
called for OGG files. Since that seems to fix the issue, make it
always call flac_process_metadata().
Since decoder_list.c does not include the libflac headers, it cannot
know whether to add the oggflac plugin to the decoder list. Solve
this by always enabling the oggflac sub-plugin, even with older
libflac versions. When the libflac API cannot support oggflac,
disable the plugin at runtime by returning "false" from its init()
method.
The "oggflac" plugin was enabled only if HAVE_FLAC_COMMON was
defined. HAVE_FLAC_COMMON however is only an automake variable, and
is never available in decoder_list.c. Make decoder_list.c depend on
HAVE_FLAC||HAVE_OGGFLAC instead.
The player did not care about the exact error value, it only checked
whether an error has occured. This could fit well into
decoder_control.state - introduce a new state "DECODE_STATE_ERROR".
At this moment the wavpack lib doesn't use the return value of the
push_back function, which has an equivalent meaning of the return
value of ungetc(). This is a lucky situation, because so far it
simply returned with 1 as a hard coded value. From now on the
function will return EOF on error. (This function makes exactly one
byte pushable back.)
There are some functions in the wavpack-mpd input streams wrapper
which had too commonly used names (especially can_seek). I prefixed
these with "wavpack_input_".
The listen.c module breaks the build because the variable name used
("sun") for the Unix domain socket part collides with something else
on an OpenSolaris system, likely Sun specific. Renaming it to _sun
(or something else of choice) fixes the build.
[mk: renamed to "s_un"]
I had this option enabled during development, but at some point, it
must have gotten lost. FAILONERROR makes the curl stream fail when
the server returns a status code 400 or higher. We are not interested
in the server's error document.
Initialize libc's locale functions. Currently, we are only interested
in LC_CTYPE (character classification), because this is what is used
by GLib's g_get_charset().
GLib provides the function g_get_filename_charsets() which determines
the file system character set. This changes MPD's fallback: GLib
prefers UTF-8 as a fallback. MPD used to fall back to ISO Latin 1.
libwavpack expects the read_bytes() stream method to fill the whole
buffer, and fails badly when we return a partial read (i.e. not enough
data available yet). This caused wavpack streams to break.
Re-implement the buffer filling loop.
Instead of manually waiting for the input stream to become ready (to
catch server errors), just read the first byte. Since the
wavpack_input has the capability to push back one byte, we can simply
re-feed it. Advantage is: decoder_read() handles everything for us,
i.e. waiting for the stream, polling for decoder commands and error
handling.
The API of mp4_load_tag() was strange: it always returned a tag
object, no matter if a tag was found in the file; the existence of a
tag was indicated with the tag_found integer reference. This flag is
superfluous, since we can simply check whether the tag is empty or
not.
Allocate the mp4ff_callback_t object on the stack. This is easier to
handle, since we don't have to free it. Incidentally, this fixes a
memory leak in mp4_load_tag().
The function decoder_read() already cares about the decoder command,
and loops until data is available. Reduced mpd_ffmpeg_read() to no
more than the decoder_read() call.
The variable "next_song" is already protected by a memory barrier.
"total_time" is not important for synchronization, and we don't need
"volatile" here.
If an input stream provides tags (e.g. from an icecast server), send
them in the decoder_data() and decoder_tag() methods. Removed the
according code from the mp3 and oggvorbis plugins - decoders shouldn't
have to care about stream tags.
This patch also adds the missing decoder_tag() invocation to the mp3
plugin.
MPD used to have a copy of the mp4ff library. Since that has been
removed, AAC suport was disabled when there was no libmp4ff. Separate
the libmp4ff test, and enable AAC support no matter if libmp4ff is
available.
The "mod" decoder plugin was being initialized lazily, but was
deinitialized unconditionally. That led to segmentation faults.
Convert mod_initMikMod() to be the global module initialization
method. The MPD core should care about lazy initialization.
Non-local songs used to have no tags. If the decoder sends us a tag,
we should incorporate it into the song struct. This way, clients can
always show the correct song name (if provided by the server).
The try_decode() method may have read some data from the stream, which
is now lost. To make this data available to other methods, get it
back by rewinding the input stream after each try_decode() invocation.
The ogg and wavpack plugins did this manually and inconsistently; this
code can now be removed.
If the source chunk has a tag, merge it into the destination chunk.
The source chunk gets deleted after that, and this is our last chance
to grab the tag.
Ogg and ffmpeg detection was disabled when the stream was not
seekable, because the detection was too expensive. Since the curl
input stream can now rewind the stream cheaply, we can re-enable
detection on streams.
During codec detection, the beginning of the stream is consumed. This
is a common operation, which takes a lot of time when handling remote
resources. To optimize this, remember the first 64 kB of a stream.
This way, we can rewind the stream without actually fetching the start
of the stream again.
Since the aac and mod plugins have told MPD that they cannot seek, MPD
will never send a SEEK command to them. Removed the SEEK comand
checks from both plugins.
Don't pass the "seekable" flag with every decoder_data() invocation.
Since that flag won't change within the file, it is enough to pass it
to decoder_initialized() once per file.
Replace all direct music_pipe struct accesses with wrapper functions.
The compiled machine code is the same, but this way, we can change
struct internals more easily.
.. and rename dc.audioFormat to dc.in_audio_format. The music pipe
does not need to know the audio format, and its former "audioFormat"
property indicated the format of the most recently added chunk, which
might be confusing when you are reading the oldest chunks.
No CamelCase in the file name. The output_buffer struct is going to
be renamed to music_pipe. There are so many buffer levels in MPD, and
calling this one "output buffer" is wrong, because it's not the last
buffer before the music reaches the output devices.
Commit 1a4a3e1f moved decoders into a static array, but failed to
enable those plugins who did not have an init() method at all.
This patch corrects the "enabled" check.
Make map_directory_child_fs() refuse the names "." and "..". This is
currently the interface where an attacker may inject a manipulated
path (through the "update" command).
"LOG_H" is a macro which is also used by ffmpeg/log.h. This is
ffmpeg's fault, because short macros should be reserved for
applications, but since it's always a good idea to choose prefixed
macro names, even for applications, we are going to do that in MPD.
Depending on MPD's umask, the file permissions of the unix socket were
too restrictive, and many clients were not able to connect. Do a
chmod(0666) on the socket, to allow everybody to connect.
Similar to libmad, libmpcdec provides samples with higher quality than
16 bit. Send 24 bit samples to MPD, which allows MPD to apply
dithering just in case the output devices are only 16 bit capable.
The conversion of integer samples was completely broken, which
presumably didn't annoy anybody because libmpcdec provides float
samples on most installations.
Its only caller in mp3_decode() just compared its value with
DECODE_BREAK. Convert that to bool, and return false if the loop
should be ended. Also eliminate some superfluous command checking
code, which was already done in the preceding while loop.
When one of several output devices failed, MPD tried to reopen it
quite often, wasting a lot of resources. This patch adds a delay:
wait 10 seconds before retrying. This might be changed to exponential
delays later, but for now, it makes the problem go away.
When the decoder exited before the buffer has grown big enough
("buffer_before_play"), the player thread waited forever. Add an
additional check which disables buffering as soon as the decoder
exits.
The local variable "play_audio_format" is updated every time the
player starts playing a new song. This way, we always know exactly
which audio format is current. The old code broke when a new song had
a different format: ob.audio_format is the format of the next song,
not of the current one - using this caused breakage for the software
volume control.
A decoder_flush() invocation was missing in the FLAC plugin, resulting
in casual assertion failures due to a wrong assumption about the last
chunk's audio format. It's much easier to remove that decoder_flush()
function and make the decoder thread call ob_flush().
Request the next song from the playlist (by clearing pc.next_song)
only if the player command is empty. If it is not, the player may be
clearing the song that has already been queued, leading to an
assertion failure.
Remember the seek_where argument and call decoder_command_finished()
immediately. This way, the player thread can continue working, and we
can receive more commands.
This also fixes several issues which resulted in broken frames,
leading to erroneos "elapsed" values: frames weren't parsed properly,
since the code was checking for command!=NONE.
size_t and long aren't 64 bit safe (i.e. files larger than 2 GB on a
32 bit OS). Use off_t instead, which is a 64 bit integer if compiled
with large file support.
When the decoder failed to start, the function do_play() returned,
still having pc.command==PLAY. This is because pc.command was reset
only when the decoder started up successfully. Add another
player_command_finished() call in the error handler.
Replaced the local variable "colon" (which had only temporary meaning)
with the variable "value". It is a pointer to the first byte of the
header value.
Instead of managing a set of method pointers in each input_stream
struct, move these into the new input_plugin struct. Each
input_stream has only a pointer to the plugin struct. Pointers to all
implementations are kept in the array "input_plugins".
MPD's HTTP client code has always been broken, no matter how effort
was put into fixing it. Replace it with libcurl, which is known to be
quite stable. This adds a fat library dependency, but only for people
who need streaming.
MPD shouldn't integrate sources of other libraries. Since libmp4ff is
part of libfaad, we should remove the old copy from src/mp4ff and link
with the current version from libfaad instead.
PA_SAMPLE_S16NE is the only sample format which is suported by both
MPD and pulseaudio. Unfortunately, pulse does not accept 24 bit
samples.
Instead of bailing out with an error message, we should tell the MPD
core to convert all samples to 16 bit for pulse.
This bug caused the audio output devices to stay open, although MPD
wasn't playing: quitDecode() resetted player_control.command, assuming
that the command was STOP. This way, player_task() didn't see the
CLOSE_AUDIO command, and the device was kept open.
Don't clear player_control.command in quitDecode().
When the audio source provides 24 bit samples, don't bother to convert
(lossily) them to 16 bit before jack's floating point conversion - go
directly from 24 bit to float.
The JACK documentation postulates that the process() callback must not
block, therefore locking is forbidden. Anyway, the old code was racy.
Remove all locks, and don't wait for more data to become available -
just send to the port what is already in the buffer.
Another partial frame fix: the silence buffer was 1020 bytes, which
had room for 127.5 24 bit stereo frames. Don't send the partial last
frame in this case.
24 bit output is as important as 16 bit output. Provide a
pcm_convert() implementation which can convert to 24 bit with as
little quality loss as possible.
The old pcm_convert_size() ignored most of the destination format,
e.g. it did not check its sample size, and assumed it is 16 bit.
Simplify and universalize it by using audio_format_frame_size().
Similar to pcm_resample_16(), implement pcm_resample_24(). The 24 bit
implementation is very similar, but it uses src_int_to_float_array()
instead of src_short_to_float_array() before sending data to
libsamplerate.
Use sizeof(sample) instead of hard-coding "2". Although we're in 16
bit right now, this will make code sharing easier when we support
other sample sizes.
libmad produces samples of more than 24 bit. Rounding that down to 16
bits using dithering makes those people lose quality who have a 24 bit
capable sound device. Send 24 bit PCM data, and let the receiver
decide whether to apply 16 bit dithering.
I added 24 bit support a while ago, but it wasn't possible to force 24
bit output. Add 24 and 8 bit to the list of allowed sample sizes.
Although 8 bit audio isn't as widely used as 24 bit, there is no
reason to exclude it.
Splitting a frame between two buffer chunks causes distortion in the
output. MPD used to assume that the chunk size 1020 would never cause
splitted frames, but that isn't the case for 24 bit stereo (127.5
frames), and even less for files with even more channels.
Many command arguments must not be negative; add a separate
parser/checker function for that. For the same reason, add
check_bool(). This eliminates two strange special cases handlers from
check_int().
Pass index arguments as unsigned integers. They must not be negative,
and even if some caller accidently passes -1, it won't pass the bound
checks (since it's now 2**32-1).
There are some integers which have a "magic" -1 value which means
"undefined" or "nothing". All others can be converted to unsigned,
since they must not contain a negative number.
Also add names for "error" and "ok". I don't like passing anonymous
integer codes around.
This is not yet complete: lots of functions (e.g. in playlist.c)
follow the same convention of -1/0, and these have to be adapted, too.
spl_list() provides an interface for enumerating all stored playlists.
This separates the internal playlist logic from the protocol specific
function lsPlaylists().
The two functions clearStoredPlaylist() and addToStoredPlaylist()
don't belong into playlist.c. clearStoredPlaylist() was a wrapper for
spl_clear(), and is converted into a CPP macro for now.
The list of commands is known at compile time. Instead of creating a
linked list on startup, we can just register all commands in a static
sorted array.
The command pointers which are passed around aren't being modified -
in fact, no command pointer must be modified once it has been added to
the commandList.
Instead of manually calling memset(0) on the pcm_convert_state struct,
client code should use a library function from pcm_utils.c. This way,
we can change the semantics of the struct easily.
Casting a pointer to some sort of integer and formatting it into a
string isn't valid. A pointer derived from this hex string won't work
reliably. Since ffmpeg doesn't provide a nice API for passing our
pointer, we have to think of a different hack: ffmpeg passes the exact
URL pointer to mpdurl_open(), and we can make this string part of a
struct. This reduces the problem to casting the string back to the
struct.
This is still a workaround, but this is "sort of portable", unless the
ffmpeg people start messing with the URL pointer (which would be valid
according to the API definition).
Since ffmpeg svn r12865, you have to include libavcodec/avcodec.h
instead of avcodec.h. This cannot be checked at compile time, instead
we have to add a check to configure.ac. Viliam's original ffmpeg
plugin was based on the newer ffmpeg library, while my Debian
installation had the older version. My attempt to correct his include
statements wasn't correct after all.
{song,dir}vec_for_each each failed to gracefully handle deleted
files when iterating through. While we were thread-safe, we
were not safe within the calling thread. If a callback we
passed caused sv->nr to shring, our index would still increment;
causing files to stay in the database.
A way to test this is to remove 10 or so contiguous songs from a
>10 song directory.
Like the songvec nr_lock, only one lock is used for all
traversals since they're rarely changed. This only
projects traversals, but not the individual structures
themselves.
Use a literal in the struct declaration, and sizeof(client->buffer)
everywhere else. Also shrink the buffer from 40 kB to 4 kB. The
buffer must only be large enough to hold one line of input, and 4 kB
is still more than enough.
When adding a local file, clients have to use the "file" URI schema
described in RFC 1738 3.10. By adding this schema to "urlhandlers", a
client can detect whether this feature is available.
By default, glibc 2.8 hides struct ucred behind the _GNU_SOURCE
macro. I don't want to enable that globally, because it may encourage
the use of non-portable functions. Test if "struct ucred" is
available, and enable _GNU_SOURCE if required.
For details about that issue, see glib's bug database:
http://sources.redhat.com/bugzilla/show_bug.cgi?id=6545
Some functions assume that a song is not in the database when it is a
remote song. Based on that, they decide whether they are responsible
for freeing the song struct. Add a special function which checks
whether a song is in the database (currently equal to song_is_file()).
GLib provides an easier API for character set conversion than iconv().
Use g_convert() / g_convert_with_fallback() for all character
conversions. We should optimize the path.h API later to return a
newly allocated buffer, so we can just pass GLib's return value.
GLib is a nice and portable utility library. We are going to use it
from now on, and eliminate a lot of duplicated code from MPD. Why
invent the wheel again and again?
Use memchr() instead of manually traversing the input buffer. Update
the client's properties after all commands have been processed. Check
for buffer overflow once.
The caller already knows the protocol family, and we can eliminate the
complicated switch statement in establishListen() if we just pass this
information. This seems more robust.
"idle" waits until something noteworthy happens on the server,
e.g. song change, playlist modified, database updated. This allows
clients to keep up to date without polling.
Added mpd.conf options for disabling automatic resamling, sample
format and channel conversion. This way, users may choose to override
ALSA's automatic resampling, and use libsamplerate instead.
This git branch has become a real MPD fork now. Time to change the
package name to the code name "mpd-mk". Set the version number to
"0.14~git" to mark this as a non-released version.
Don't follow relative symlinks which point into the music directory.
This allows you to organize music with symbolic links, without MPD
managing separate copies of each song.
The mapper library maps directory and song objects to file system
paths. With this central library, the code mixture in path.c should
be cleaned up, and we will be able to add neat features like aliasing.
isMusic() used to be a very inefficient function: with every
invocation, it did another stat() on the specified file. There is
only one caller, do the stat() there manually and use hasMusicSuffix()
instead of isMusic().
By always creating the parent directory, we can use delete_name_in()
without further lookups. The parents which may non exist will be
pruned later. An update request for a non-existing or empty directory
should be quite unusual, so this doesn't add any measurable overhead.
In order to optimize buffer usage, pass only the base file name to
updateInDirectory(). This way, updateInDirectory() may choose when to
allocate a larger buffer for the full path.
It is invalid to pass a path with the wrong dirname to dirvec_find().
To be able to find a subdirectory only by its basename, compare only
the basename of both paths.
The only caller of deletePlaylist() appends PLAYLIST_FILE_SUFFIX, so
we can be sure it's already there. We don't need to stat the file,
since unlink() does all the checking.
Commit 80a2c937 broke resume after pause: it cleared the
input_audio_format when it attempted to simplify a complicated
expression. Don't clear it, just assign input_audio_format if a new
format was specified.
We only need to lock sv->nr changes to prevent traversals ( why
it's called "nr_lock"). free(3) is a "slow" function on my
system; so we can avoid unnecessarily holding a lock long for
longer than needed.
If the sample format isn't supported by the device (i.e. 24 bit on
low-end sound chips), fall back to 16 bit output. There is code in
pcm_utils.c which converts PCM data to 16 bit.
Convert any number of channels to stereo. In fact, this isn't really
stereo, it's rater mono blown up to stereo. This patch should only
make it possible to play 5.1 files at all; "real" conversion to stereo
should be implemented, but for now, this is better than nothing.
In order to be able to deal with non-trivial conversions,
pcm_convertChannels() needs to know both the input and the output
channel count. Simplify buffer allocation in that function.
Moved code from pcm_convertChannels() to pcm_convert_channels_1_to_2()
and pcm_convert_channels_2_to_1(). Improved the quality of
pcm_convert_channels_2_to_1() by calculating the arithmetic mean value
of both samples.
buffered_before_play was copied to struct player because it was used
to disable buffering when seeking. Instead of mainaining a copy of
this number, move just the flag to the player struct.
Renamed audio_configFormat to configured_audio_format. Renamed
audio_buffer.format to input_audio_format. Simplified its
initialization in openAudioDevice().
audio.c maintained one of MPD's many layers of audio buffers. It was
without any benefit, since playAudio() can simply send the source
buffer directly to the audio output plugin.
QUEUE adds a new song to the player's queue. CANCEL clears the queue.
These two commands replace the old and complex queueState and
queueLockState code.
Simplify and merge several if clauses before the clearPlayerQueue()
invocation. Call clearPlayerQueue() only if a song is actually
queued; add an assertion for that in clearPlayerQueue().
This variable is superfluous, it is only used to copy its value to
player_control.totalTime. Since the original source of this value
(song->tag->time) will still be available at this point, we can safely
remove fileTime.
Revert e4f5d6bd "re-enable-nonblocking, but sleep if busy".
Non-blocking mode with manual sleeping doesn't help at all (by the
way, the patch should have used snd_pcm_wait() instead of
my_usleep()). ALSA knows much more about the hardware quirks, so we
just let it do the job.
Leftover from the output API changes: oss_open_default() was changed
to return a void*, but it still returned "0" to report success.
Report the OssData pointer instead.
The decoder was woken up after each chunk which had been played. That
caused a lot of superfluous context switches. Wake up the decoder
only when a certain amount of the buffer has been consumed. This
formula is somewhat arbitrary, and has to be proven experimentally.
The mp3 plugin did not use the MAD_NCHANNELS() value correctly: when a
stream was not stereo, it was assumed to be mono, although the correct
number was passed to MPD. libmad doesn't support more than 2
channels, but this change allows gcc to optimize its inlining
strategy.
The dithering function audio_linear_dither() worked for signed 16 bits
only anyway, having a variable "bits" just disables important gcc
optimizations.
A frame contains one sample per channel, thus it is sample_size *
channels. This patch includes some cleanup for various locations
where the sample size for 24 bit audio was still 3 bytes (instead of
4).
There is only once update thread at a time. Make the "modified" flag
global and remove the return values of most functions. Propagating an
error is only useful for updateDirectory(), since updateInDirectory()
will delete failed subdirectories.
The documentation for directory_update_init() was incorrect: a job ID
must be positive, not non-negative. If the update queue is full and
no job was created, it makes more sense to return 0 instead of -1,
because it is more consistent with the return value of isUpdatingDB().
pthread_join() expects a "pointer to a pointer" parameter, but it got
a "pointer to an enum". On AMD64, an enum is smaller than a pointer,
leading to a buffer overflow.
In updateInDirectory(), add new directories immediately and
delete them when they turn out to be empty. This simplifies the code
and allows us to eliminate addSubDirectoryToDirectory().
If the user requests database update during startup, call
directory_update_init(). This should be changed to fully asynchronous
update later.
For this to work, main_notify has to be initialized before db_init().
The algorithm in addDirectoryPathToDB() can be simplified further if
it is combined with the function addParentPathToDB(). Since there is
no other caller of addDirectoryPathToDB(), we can do that. This saves
another large stack buffer.
This recursive function is very dangerous because it allocates a large
buffer on the stack in every iteration. That may be misused to
generate a stack overflow.
When a directory failed to update, it was removed from the database,
without freeing all children and songs (memory leak), and without
locking (race condition). Introduce the functions clear_directory()
and delete_directory(), which do both.
Don't use db_get_directory() and traverse the full path with every
directory being loaded. Just see if the current parent contains the
entry. Everything else would be invalid anyway..
A manipulated database could trigger an assertion failure, because the
parent didn't match. Do a proper check if the new directory is within
the parent's. This uses FATAL() to bail out, so MPD still dies, but
it doesn't crash.
Remove clutter from directory.c. Everything which saves or loads
to/from the hard disk goes to directory_save.c, and code which sends
directory information to the client is moved into directory_print.c.
Having an array with disabled entries sucks. Removed that
DISABLED_SHOUT_ENCODER_PLUGIN macro, and fill the plugin list only
with plugins which are actually enabled. This should be done for all
plugin types.
"volume" was passed as an unsigned integer, which is correct. It's
just that when it was multiplied with the sample value, the whole
operation was changed to unsigned, breaking the algorithm (and Qball's
ears). Internally change "volume" to signed.
With commit 6dcd7fea (if I am not mistaken) the error returned when
you try to save to an existing playlist is wrong. Instead of
MPD_ACK_ERROR_EXIST, MPD_ACK_ERROR_NO_EXIST is returned. This is
obviously wrong and breaks gmpc.
Commit 0bfe7802 broke update for new files in the root directory,
because music_root->path was an empty string and not NULL. There were
some NULL tests missing. Change them to !isRootDirectory(path)
instead of path!=NULL.
Taming the directory.c monster, part II: move the database management
stuff to database. directory.c should only contain code which works
on directory objects.
Instead of returning 0 or -1, return true on success and false on
failure. This seems more natural, and when the C library was
designed, there was no "bool" data type.
Provide separate constructors for creating a remote song, a local
song, and one for loading data from a song file. This way, we can add
more assertions.
exploreDirectory() duplicates some code in updateDirectory(). Merge
both functions, and use directory_is_empty() to determine whether
update or explore mode should be used.
The source directory.c mixes several libraries: directory object
management, database management and database update, resulting in a
1000+ line monster. Move the whole database update code to update.c.
Having all functions as static (non-inline) functions generates GCC
warnings, and duplicates binary code across several object files.
Most of dirvec's methods are too complex for becoming inline
functions. Move them all to dirvec.c and publish the prototypes in
dirvec.h.
pthread_cond_wait() may wake up spuriously. To prevent superfluous
state checks, loop until the "pending" flag becomes true. Removed the
dangerous assertion.
This makes the update code thread-safe and doesn't penalize
the playlist code by complicating it with complicated and
error-prone locks (and the associated overhead, not everybody
has a thread-implementation as good as NPTL).
The update task blocks during the delete; but the update task is
a slow task anyways so we can block w/o people caring too much.
This was also our only freeSong call site, so remove that
function.
Note that deleting entire directories is not fully thread-safe,
yet; as their traversals are not yet locked.
Only one lock is used for all songvec traversals since
they're rarely changed. Also, minimize lock time and
release it before calling iterator functions since they
may block (updateSongInfo => stat/open/seek/read).
This lock only protects songvecs (and all of them) during
traversals; not the individual song structures themselves.
* Add missing headers in Makefile.am
* remove mp4ff.dsp (Win32 crap)
* Add scripts, m4, bs, autogen.sh to allow for hotfixes by the
SCM-challenged. (downloading the source via git is NOT a
lightweight operation for everybody).
We already know if a song is a URL or not based on whether it
has parentDir defined or not. Hopefully one day in the future
we can drop HTTP support from MPD entirely when an HTTP
filesystem comes along and we can access streams via open(2).
The "packed" attribute may have negative side effects on performance.
Remove the "packed" attribute, and increase the size of "song.url" to
a multiple of the machine word size.
This got broken when listHandlerFunc was removed. Since we no
longer need it and it's confusing, remove processCommandInternal
and just use process_command.
Instead of allocating a new one, just reuse an existing
one if one is found when rereading the DB. This is a small
makes the previous commit work on subdirectories
of the root music directory.
[1] "song: better handling of existing songs when rereading DB"
commands should really not behave differently if they're issued
inside a command list or not; so stop having special handler
functions to deal with them. "update" was the only command
that used this functionality and I changed that in the last
commit to serialize access.
Now the "update" command can be issued multiple times regardless
of whether the client is in list mode or not.
We serialize the update tasks to prevent updates from trampling
over each other and will spawn another update task
once the current one is finished updating and reaped.
Right now we cap the queue size to 32 which is probably enough (I
bet most people usually run update with no argument anyways);
but we can make it grow/shrink dynamically if needed. There'll
still be a hard-coded limit to prevent DoS attacks, though.
Add support for 24 bit PCM samples to all functions. Note that
pcm_convertAudioFormat() converts 24 bit samples to 16 bit; to
preserve full quality, support for "real" 24 bit conversion should be
added.
Moved code into separate bit specific functions:
- pcm_volumeChange() -> pcm_volume_change_X()
- pcm_add() -> pcm_add_X()
- pcm_convertTo16bit() -> pcm_convert_8_to_16()
pcm_mix() might overflow the destination buffer if it is smaller than
the second buffer. This is ok because the physical buffer size passed
by cross_fade_apply() is always big enough, but clutters pcm_mix()
with complicated length checks and contains a dangerous buffer
overflow pitfall. Simplify pcm_mix()/pcm_add() and pass only the
smaller buffer size; let cross_fade_apply() do the memcpy().
pause() puts the audio output into pause mode: if supported, it may
perform a special action, which keeps the device open, but does not
play anything. Output plugins like "shout" might want to play silence
during pause, so their clients won't be disconnected. Plugins which
do not support pausing will simply be closed, and have to be reopened
when unpaused.
This pach includes an implementation for the shout plugin, which
sends silence chunks.
The function audio_output_is_pending() returns whether there is a
pending command. This is useful for output plugins as a break
condition for longer loops.
The old struct initializers are error prone and don't allow moving
elements around. Since we are going to overhaul some of the APIs
soon, it's easier to have all implementations use C99 initializers.
Since we use a C99 compiler now, we can assert that the C99 standard
headers are available, no need for complicated compile time checks.
Kill mpd_types.h.
Having an enum type is much nicer than an anonymous integer plus CPP
macros. Note that the old code didn't save any space by declaring the
variable 8 bit, due to padding.
Seeing the "mpd_" prefix _everywhere_ is mind-numbing as the
mind needs to retrain itself to skip over the first 4 tokens of
a type to get to its meaning. So avoid having extra characters
on my terminal to make it easier to follow code at 2:30 am in
the morning.
Please report any new issues you may come across on Free
toolchains. I realize how difficult it can be to build/maintain
cross-compiling toolchains and I have no intention of forcing
people to upgrade their toolchains to build mpd.
Tested with gcc 2.95.4 and and gcc 4.3.1 on x86-32.
tfing wrote:
> I have quite some files with an empty album tag as they do not come
> from a particular album.
>
> If I want to look for those files and browse them, this happens:
> :: nc localhost 6600
> OK MPD 0.12.0
> find album ""
> ACK [2@0] {find} too few arguments for "find"
>
> I'd like to be able to browse those files in a client like gmpc.
> So these 2 items would have to be developed:
> - list album should report that some files have an empty tag
> - it should be possible to search for an empty tag with the find command
Patch-by: Marc Pavot
ref: http://musicpd.org/mantis/view.php?id=464
This only breaks "update" under list command mode and
no other commands. This can be done more optimally
without the extra heap allocation via xstrdup(); but is
uncommon enough to not matter.
It was a huge confusing mess of parameter passing around
and around. Add a few extra assertions to ensure we're
handling parent/child relationships properly.
This is like basename(3) but with predictable semantics independent
of C library or build options used. This is also much more strict
and does not account for trailing slashes (mpd should never deal with
trailing slashes on internal functions).
If we updated the mpd metadata database; then there's a chance
some of those songs in the playlist will have updated metadata.
So be on the safe side and increment the playlist version number
if _any_ song changed (this is how all released versions of mpd
did it, too).
This bug was introduced recently when making "update" threaded.
Thanks to stonecrest for the bug report.
Make the code more readable by moving the range checks to pcm_range().
gcc does quite a good job at optimizing it: the resulting binary is
exactly the same, although it contains a parametrized shift instead of
hard-coded boundaries.
There was a known deadlocking bug in the notify library: when the
other thread set notify->pending after the according check in
notify_wait(), the latter thread was deadlocked. Resolve this by
synchronizing all accesses to notify->pending with the notify object's
mutex. Since notify_signal_sync() was never used, we can remove it.
As a consequence, we don't need notify_enter() and notify_leave()
anymore; eliminate them, too.
During debugging, I found a deadlock between flushAudioBuffer() and
the audio_output_task(): audio_output_task() didn't notice that there
is a command, and flushAudioBuffer() waited forever in notify_wait().
I am not sure yet what is the real cause; work around this for now by
waking up non-finished audio outputs in every iteration.
Due to a merge error, I broke the function handleUpdate(). It did not
do anything for the global update, and it did not send a proper
response to the client. This patch fixes both bugs.
To check whether a device is really on or off, we should rather check
audio_output.open, instead of managing another variable. Wrap
audio_output.open in the inline function audio_output_is_open() and
use it instead of DEVICE_ON and DEVICE_OFF.
Send an output buffer to all output plugins at the same time, instead
of waiting for each of them separately. Make several functions
non-blocking, and introduce the new function audio_output_wait_all()
to synchronize with all audio output threads.
We have eliminated direct accesses to the audio_output struct from
the all output plugins. Make it opaque for them, and move its real
declaration to output_internal.h, similar to decoder_internal.h.
Pass the opaque structure to plugin.init() only, which will return the
plugin's data pointer on success, and NULL on failure. This data
pointer will be passed to all other methods instead of the
audio_output struct.
The JACK output plugin needs to reset its "opened" flag when the JACK
server fails. To prevent it from accessing the audio_output struct
directly introduce the API function audio_output_closed().
Reduce direct accesses to the audio_output struct from the plugins:
this time, eliminate all accesses to audio_output.name. The name is
required by some plugins for log messages.
Pass the globally configured audio_format as a const pointer to
plugin.init(). plugin.open() gets a writable pointer which contains
the audio_format requested by the plugin. Its initial value is either
the configured audio_format or the input file's audio_format.
To keep I/O nastiness and latencies away from the core, move the audio
output code to a separate thread, one per output. The thread is
created on demand, and currently runs until mpd exits.
Since flacSendChunk() is a trivial function and is only used in one
location, move the code there. The advantage is that calling
decoder_data() directly returns the decoder_command value, so we can
eliminate one decoder_get_command() call.
Support for bit rates except 16 bits (and 8 bits on little endian) has
always been broken. Since we added optimized functions for 8, 16,
24/32 bits, we can remove the generic flac_convert() function.
Instead of removing it, convert it to a wrapper function for
flac_convert_*().
flac_convert_16() runs a lot faster than the generic (and quite buggy)
function flac_convert(). flac_convert_16() is only used for
non-stereo files, since there is already flac_convert_stereo16().
By mistake, I casted the sample value to uint16_t, which is wrong.
This patch simplifies the code by using a int16_t pointer instead of
casting to int16_t* every time.
There is still a lot of duplicated code in flac_plugin.c and
oggflac_plugin.c. Move code from flac_plugin.c to _flac_common.c, and
use the new function flac_common_write() also in oggflac_plugin.c,
porting lots of optimizations over to it.
The inline function audio_format_sample_size() calculates how many
bytes each sample consumes. This function already takes into account
that 24 bit samples are 4 bytes long, not 3.
Instead of letting ALSA block for us (and potentially allowing
something stupid on certain hardware or drivers), we do the
sleeping ourselves. We calculate the sleep to be a fraction of
period_time to avoid oversleeping (and thus audible skipping).
A lot of the preparation was needed (and done in previous
months) in making update thread-safe, but here it is.
This was the first thing I made work inside a thread when I
started mpd-uclinux many years ago, and also the last thing I've
done in mainline mpd to work inside a thread, go figure.
pthreads with our existing signal blocking/handling is broken,
for now just sleep a bit in the child to prevent the CHLD handler
from being called too early. Also, improve error reporting when
handling SIGCHLD by storing the status to be called in the main
task (which can be logged, since we can't do logging inside the
sig handler).
Our linked-list implementation is wasteful and the
SongList isn't modified enough to benefit from being a linked
list. So use a more compact array of song pointers which
saves ~200K on a library with ~9K songs (on x86-32).
It hasn't been used in many years
commit 3a89afdd80
Author: Warren Dukes <warren.dukes@gmail.com>
Date: Sat Nov 20 20:28:32 2004 +0000
remove --update-db option
(SVN r2719)
This allows us to avoid the nasty repetition in strncmp(foo,
bar, strlen(foo)). We'll miss out on the compiler optimizing
strlen() into sizeof() - 1 for string literals for this; but we
don't use this it for performance-critical functions anyways...
This should save a few thousand ops. Not worth it to malloc
for such a small (3-words on 32-bit ARM and x86) structures.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
The function decodeFirstFrame() allocates memory based on data from
the mp3 header. This can make the buffer size allocation overflow, or
lead to a DoS attack with a very large buffer. Cap this buffer at 8
million frames, which should really be enough for reasonable files.
The assertion on "!client_is_expired(client)" was wrong, because
writing the command response may cause the client to become expired.
Replace that assertion with a check.
A crafted mp4 file could cause an integer overflow in mp4_decode
function in src/inputPlugins/mp4_plugin.c. mp4ff_num_samples()
function returns some tainted value. sizeof(float) * numSamples is an
integer overflow operation if numSamples is too huge, so xmalloc will
allocate a small memory region. I constructe a mp4 file, and use
faad2 to open the file. mp4ff_num_samples() returns -1. So I think mpd
bears from the same problem.
Since the buffer size is known at compile time, we can save an
indirection by declaring it as a char array instead of a pointer.
That saves an extra allocation, and we can calculate with the
compile-time constant sizeof(data) instead of the attribute "max_len".
Shout encoder plugins are known at compile time. There is no reason
to use a complex data structure as "List" to manage them at runtime -
just put the pointers into a static array.
[mk: moved this patch after "Refactor and cleanup of shout Ogg and MP3
audio outputs". The original commit message follows, although it is
outdated:]
Creation of shout_mp3 audio output plugin. Basically I just copied the
existing shout plugin and replaced ogg with lame. Uses lame for mp3
encoding. Next step is to pull common functionality out of each shout
plugin and share it between them.
Configuration options for "shout_mp3" are the same as for "shout".
I've perhaps gone a bit overboard, but here's the current rundown:
Both Ogg and MP3 use the "shout" audio output plugin. The shout audio
output plugin itself has two new plugins, one for the Ogg encoder,
and another for the MP3 (LAME) encoder.
Configuration for an Ogg stream doesn't change. For an MP3 stream,
configuration is the same as Ogg, with two exceptions. First, you must
specify the optional "encoding" parameter, which should be set to "mp3".
See mpd.conf(5) for more details. Second, the "quality" parameter is
reversed for LAME, such that 1 is high quality for LAME, whereas 10 is
high quality for Ogg.
I've decomposed the code so that all libshout related operations
are done in audioOutput_shout.c, all Ogg specific functions are in
audioOutput_shout_ogg.c, and of course then all LAME specific functions
are handled in audioOutput_shout_mp3.c.
To develop encoder plugins for the shout audio output plugin, I basically
just mimicked the plugin system used for audio outputs. This might be
overkill, but hopefully if anyone ever wants to support some other sort
of stream, like maybe AAC, FLAC, or WMA (hey it could happen), they will
hopefully be all set.
The Ogg encoder is slightly less optimal under this configuration.
It used to send shout data directly out of its ogg_page structures. Now,
in the interest of encapsulation, it copies the data from its ogg_page
structures into a buffer provided by the shout audio output plugin (see
audioOutput_shout_ogg.c, line 77.) I suspect the performance impact
is negligible.
As for metadata, I'm pretty sure they'll both work. I wrote up a test
scaffold that would create a fake tag, and tell the plugin to send it
out to the stream every few seconds. It seemed to work fine. Of course,
if something does break, I'll be glad to fix it.
Lastly, I've renamed lots of things into snake_case, in keeping with
normalperson's wishes in that regard.
[mk: moved the MP3 patch after this one. Splitted this patch into
several parts; the others were already applied before this one. Fixed
a bunch GCC warnings and wrong whitespace modifications. Made it
compile with mpd-mk by adapting to its prototypes]
Support sending metadata to a shout server using shout_metadata_new()
and shout_metadata_add(). The Ogg Vorbis encoder does not support
this currently.
[mk: this patch was separated from Eric's patch "Refactor and cleanup
of shout Ogg and MP3 audio outputs", I added a description]
Preparing the merge of Eric Wollesen's patch "Refactor and cleanup of
shout Ogg and MP3 audio outputs": we declare one of the struct types
here, to make the merge smoother.
The Ogg encoder is slightly less optimal under this configuration. It
used to send shout data directly out of its ogg_page structures. Now,
in the interest of encapsulation, it copies the data from its ogg_page
structures into a buffer provided by the shout audio output plugin
(see audioOutput_shout_ogg.c, line 77.) I suspect the performance
impact is negligible.
[mk: this patch and its description was separated from Eric's patch
"Refactor and cleanup of shout Ogg and MP3 audio outputs"]
Begin dividing audioOutput_shout.c: move everything OGG Vorbis related
to audioOutput_shout_ogg.c. The header audioOutput_shout.h has to
keep its dependency on vorbis/vorbisenc.h, because it needs the vorbis
encoder types.
For this patch, we have to export several internal functions with
generic names to the ABI; these will be removed later when the encoder
plugin patches are merged.
Remove unused code which is in comments. Remove that comment about
"stolen code", since the plugin has changed much, and it isn't obvious
which parts are derived.
If the output device is already open, it may have modified
outAudioFormat; in this case, outAudioFormat is still valid, and does
not need an overwrite.
As long as the device isn't open, both attributes are not used. Since
they will both be initialized in audio_output_open(), we do not need
the initialization in audio_output_init().
Storing pointers to immutable audio_format structs isn't worth it,
because the struct itself isn't much larger than the pointer. Since
the shout plugin requires the user to configure a fixed audio format,
we can simply copy it in myShout_initDriver().
Save one allocation, since the whole audio_format struct is nearly the
same size as the pointer to it. Check audio_format_defined(af)
instead of af!=NULL.
free(NULL) isn't explicitly forbidden, but isn't exactly good style.
Check the rare case that the audio buffer isn't initialized yet in
closeAudioDevice(). In this case, we also don't have to call
flushAudioBuffer().
To make openAudioDevice() smaller and more readable, move code to a
static function. Also don't use realloc(), since the old value of the
buffer isn't needed anymore, saving a memcpy().
There are too many static variables in audio.c - organize all
properties of the audio buffer in a struct. The current audio format
is also a property of the buffer, since it describes the buffer's
data format.
audio_format_clear() sets an audio_format struct to an cleared
(undefined) state, which is both faster and smaller than memset(0).
audio_format_defined() checks if the audio_format struct actually has
a defined value (i.e. non-zero). Both can be used to avoid pointers
to audio_format, replacing the "NULL" value with an "undefined"
audio_format.
Since the caller chain doesn't care about the return value (except for
COMMAND_RETURN_KILL, COMMAND_RETURN_CLOSE), just return 0 if there is
nothing special. This saves one local variable initialization, and
one access to it.
Also remove one unreachable "return 1" from client_read().
Don't close the client within client_process_line(), return
COMMAND_RETURN_CLOSE instead. This is the signal for the caller chain
to actually close it. This makes dealing with the client pointer a
lot safer, since the caller always knows whether it is still valid.
The "!src" check in copyAudioFormat() used to hide bugs - one should
never pass NULL to it. There is one caller which might pass NULL, add
a check in this caller.
Instead of doing mempcy(), we can simply assign the structures, which
looks more natural.
The way we used non-blocking mode was HORRIBLE.
It was non-blocking to ALSA, but we end up blocking in a busy
loop that does absolutely NOTHING but retry. We don't check
for playback cancellation (like we do in decoders) or anything.
This is seriously broken and I can imagine it affects people on
fast CPUs more because we do asynchronous output buffering and
our ALSA device will always have data ready.
This is safer than the patch in
http://www.musicpd.org/mantis/view.php?id=1542
with multiple audio outputs enabled.
Sadly, I only noticed that patch/problem when I googled for
"snd_config_update_free_global"
Apparently snd_pcm_hw_params_can_resume() can return false even
though my hardware does in fact support resuming. So stop
carrying that value in the canResume flag and just try to resume
when we're in the suspended state; falling back to
snd_pcm_prepare only if resuming fails. libao does something
similar on resume, too.
While we're at it, use the E() macro which will enable us to
have better error reporting.
[mk: remove the E() macro stuff]
With a large music database, the linear string collection in
tagTracker.c becomes very slow. We implemented that in a
quick'n'dirty fashion when we removed tree.c, and now we rewrite it
using the fast hashed string set.
"struct strset" is a hashed string set: you can add strings to this
library, and it stores them as a set of unique strings. You can get
the size of the set, and you can enumerate through all values.
This will be used to replace the linear tagTracker library.
Instead of having to register each output plugin, store them
statically in an array. This eliminates the need for the List library
here, and saves some small allocations during startup.
Due to clumsy layout, the audio_format struct took 12 bytes. Move the
"channels" to the end, so it can be merged into the same 32 bit slot
as "bits", which reduces the struct size to 8 bytes.
print_playlist_result() had an assert(0) at the end, in case there was
an invalid result value. With NDEBUG, this resulted in a function not
returning a value - add a dummy "return -1" at the end to keep gcc
quiet.
Since all callers of song_id_exists() will map it to a song position
after the check, introduce a new function called song_id_to_position()
which performs both the check and the map lookup, including nice
assertions.
volatile provides absolutely no guarantee thread-safety in SMP
environments. volatile was designed to access memory locations
in peripheral hardware directly; not for SMP. If volatile is
needed to work properly on SMP, then it is only hiding subtle
bugs.
volatile only prevents the /compiler/ from making optimizations
when accessing variables. CPUs do their own optimizations at
runtime so it cannot guarantee registers of CPUs are flushed
to memory cache-coherent access on different CPUs.
Furthermore, the thread-communication via condition variables
between threads sharing audio formats already results in memory
barriers.
The tag pool is a shared global resource that is infrequently
modified. However, it can occasionally be modified by several
threads, especially by the metadata_pipe for streaming metadata
(both reading/writing).
The bulk tag_item pool is NOT locked as currently only the
update thread uses it.
Trying to read or remember
"tag->numOfItems * sizeof(*tag->items)"
requires too much thinking and mental effort on my part.
Also, favor "sizeof(struct mpd_tag)" over "sizeof(*tag->items)"
because the former is easier to read and follow, even though
the latter is easier to modify if the items member changes
to a different type.
The previous patch enabled these warnings. In Eric's branch, they
were worked around with a generic deconst_ptr() function. There are
several places where we can add "const" to pointers, and in others,
libraries want non-const strings. In the latter, convert string
literals to "static char[]" variables - this takes the same space, and
seems safer than deconsting a string literal.
All callers of fdprintf() have been converted to client_printf() or
fprintf(); it is time to remove this clumsy hack now. We can also
remove client_print() which took a file descriptor as parameter.
Now that we have removed all invocations of client_get_fd(), we can
safely remove this transitional function. All access to the file
descriptor is now hidden behind the interface declared in client.h.
The shared code in showPlaylist() isn't worth it, because we aim to
remove fdprintf(). Duplicate this small function, and enable stdio
buffering for saved playlists.
The function loadPlaylist() wants to report incremental errors to the
client, for this reason we cannot remove its protocol dependency right
now. Instead, make it use the client struct instead of the raw file
descriptor.
Don't pass the raw file descriptor around. This migration patch is
rather large, because all of the sources have inter dependencies - we
have to change all of them at the same time.
Pass the client struct to CommandHandlerFunction and
CommandListHandlerFunction. Most commands cannot take real advantage
of that yet, since most of them still work with the raw file
descriptor.
These two functions take a client struct instead of the file
descriptor. We will now begin passing the client struct around
instead of a raw file descriptor (which needed a linear lookup in the
client list to be useful).
This patch continues the work of the previous patch: don't pass a file
descriptor at all to traverseAllIn(). Since this fd was only used to
report "directory not found" errors, we can easily move that check to
the caller. This is a great relief, since it removes the dependency
on a client connection from a lot of enumeration functions.
Database traversal should be generic, and not bound to a client
connection. This is the first step: no file descriptor for the
callback functions forEachSong() and forEachDir(). If a callback
needs the file descriptor, it has to be passed in the void*data
pointer somehow; some callbacks might need a new struct for passing
more than one parameter. This might look a bit cumbersome right now,
but our goal is to have a clean API.
Continuing the effort of removing protocol specific calls from the
core libraries: let the command.c code call commandError() based on
PlaylistInfo's return value.
The playlist library shouldn't talk to the client if possible.
Introduce the "enum playlist_result" type which the caller
(i.e. command.c) may use to generate an error message.
Client's input values should be validated by the command
implementation, and the core libraries shouldn't talk to the client
directly if possible. Thus, setPlaylistRepeatStatus() and
setPlaylistRandomStatus() don't get the file descriptor, and cannot
fail (return void).
The function valid_playlist_name() checks the name, but it insists on
reporting an eventual error to the client. The new function
is_valid_playlist_name() is more generic: it just returns a boolean,
and does not care what the caller will use it for. The old function
valid_playlist_name() will be removed later.
Currently, when the tag cache is being serialized to hard disk, the
stdio buffer is flushed before every song, because tag_print.c
performs unbuffered writes on the raw file descriptor. Unfortunately,
the fdprintf() API allows buffered I/O only for a client connection by
looking up the client pointer owning the file descriptor - for stdio,
this is not possible. To re-enable proper stdio buffering, we have to
duplicate the tag_print.c code without fprintf() instead of our custom
fdprintf() hack. Add this duplicated code to tag_save.c.
Move everything which dumps song information (via tag_print.c) to a
separate source file. song_print.c gets code which writes song data
to the client; song_save.c is responsible for serializing songs from
the tag cache.
Based on client_puts(), client_printf() is the successor of
fdprintf(). As soon as all fdprintf() callers have been rewritten to
use client_printf(), we can remove fdprintf().
client_write() writes a buffer to the client and buffers it if
required. client_puts() does the same for a C string. The next patch
will add more tools which will replace fdprintf() later.
clearMpdTag could be called on a tag that was still in a
tag_begin_add transaction before tag_end_add is called. This
was causing free() to attempt to operate on bulk.items; which is
un-free()-able. Now instead we unmark the bulk.busy to avoid
committing the tags to the heap only to be immediately freed.
Additionally, we need to remember to call tag_end_add() when
a song is updated before we NULL song->tag to avoid tripping
an assertion the next time tag_begin_add() is called.
Since client->fd==-1 has become our "expired" flag, it may already be
-1 when client_close() is called. Don't assert that it is still
non-negative, and call client_set_expired() instead.
During the tag library refactoring, the shout plugin was disabled, and
I forgot about adapting it to the new API. Apply the same fixes to
the oggflac decoder plugin.
While parsing the tag cache, don't allocate the directory name from
the heap, but copy it into a buffer on the stack. This reduces heap
fragmentation by 1%.
If many tag_items are added at once while the tag cache is being
loaded, manage these items in a static fixed list, instead of
reallocating the list with every newly created item. This reduces
heap fragmentation.
Massif results again:
mk before: total 12,837,632; useful 10,626,383; extra 2,211,249
mk now: total 12,736,720; useful 10,626,383; extra 2,110,337
The "useful" value is the same since this patch only changes the way
we allocate the same amount of memory, but heap fragmentation was
reduced by 5%.
Try to detect if the string needs Latin1-UTF8 conversion, or
whitespace cleanup. If not, we don't need to allocate temporary
memory, leading to decreased heap fragmentation.
At several places, we create temporary copies of non-null-terminated
strings, just to use them in functions like validUtf8String(). We can
save this temporary allocation and avoid heap fragmentation if we
add a length parameter instead of expecting a null-terminated string.
Since the inline function cannot modify its caller's variables (which
is a good thing for code readability), the new string pointer is the
return value. The resulting binary should be the same as with the
macro.
The new source tag_pool.c manages a pool of reference counted tag_item
objects. This is used to merge tag items of the same type and value,
saving lots of memory. Formerly, only the value itself was pooled,
wasting memory for all the pointers and tag_item structs.
The following results were measured with massif. Started MPD on
amd64, typed "mpc", no song being played. My music database contains
35k tagged songs. The results are what massif reports as "peak".
0.13.2: total 14,131,392; useful 11,408,972; extra 2,722,420
eric: total 18,370,696; useful 15,648,182; extra 2,722,514
mk f34f694: total 15,833,952; useful 13,111,470; extra 2,722,482
mk now: total 12,837,632; useful 10,626,383; extra 2,211,249
This patch set saves 20% memory, and does a good job in reducing heap
fragmentation.
The value is stored in the same memory allocation as the tag_item
struct; this saves memory because we do not store the value pointer
anymore. Also remove the getTagItemString()/removeTagItemString()
dummies.
This patch makes MPD consume much more memory because string pooling
is disabled, but it prepares the next bunch of patches. Replace the
code in tagTracker.c with naive algorithms without the tree code. For
now, this should do; later we should find better algorithms,
especially for getNumberOfTagItems(), which has become wasteful with
temporary memory.
