Seeing the "mpd_" prefix _everywhere_ is mind-numbing as the
mind needs to retrain itself to skip over the first 4 tokens of
a type to get to its meaning. So avoid having extra characters
on my terminal to make it easier to follow code at 2:30 am in
the morning.
Please report any new issues you may come across on Free
toolchains. I realize how difficult it can be to build/maintain
cross-compiling toolchains and I have no intention of forcing
people to upgrade their toolchains to build mpd.
Tested with gcc 2.95.4 and and gcc 4.3.1 on x86-32.
With a large music database, the linear string collection in
tagTracker.c becomes very slow. We implemented that in a
quick'n'dirty fashion when we removed tree.c, and now we rewrite it
using the fast hashed string set.
The tag pool is a shared global resource that is infrequently
modified. However, it can occasionally be modified by several
threads, especially by the metadata_pipe for streaming metadata
(both reading/writing).
The bulk tag_item pool is NOT locked as currently only the
update thread uses it.
Trying to read or remember
"tag->numOfItems * sizeof(*tag->items)"
requires too much thinking and mental effort on my part.
Also, favor "sizeof(struct mpd_tag)" over "sizeof(*tag->items)"
because the former is easier to read and follow, even though
the latter is easier to modify if the items member changes
to a different type.
clearMpdTag could be called on a tag that was still in a
tag_begin_add transaction before tag_end_add is called. This
was causing free() to attempt to operate on bulk.items; which is
un-free()-able. Now instead we unmark the bulk.busy to avoid
committing the tags to the heap only to be immediately freed.
Additionally, we need to remember to call tag_end_add() when
a song is updated before we NULL song->tag to avoid tripping
an assertion the next time tag_begin_add() is called.
If many tag_items are added at once while the tag cache is being
loaded, manage these items in a static fixed list, instead of
reallocating the list with every newly created item. This reduces
heap fragmentation.
Massif results again:
mk before: total 12,837,632; useful 10,626,383; extra 2,211,249
mk now: total 12,736,720; useful 10,626,383; extra 2,110,337
The "useful" value is the same since this patch only changes the way
we allocate the same amount of memory, but heap fragmentation was
reduced by 5%.
Try to detect if the string needs Latin1-UTF8 conversion, or
whitespace cleanup. If not, we don't need to allocate temporary
memory, leading to decreased heap fragmentation.
At several places, we create temporary copies of non-null-terminated
strings, just to use them in functions like validUtf8String(). We can
save this temporary allocation and avoid heap fragmentation if we
add a length parameter instead of expecting a null-terminated string.
Since the inline function cannot modify its caller's variables (which
is a good thing for code readability), the new string pointer is the
return value. The resulting binary should be the same as with the
macro.
The new source tag_pool.c manages a pool of reference counted tag_item
objects. This is used to merge tag items of the same type and value,
saving lots of memory. Formerly, only the value itself was pooled,
wasting memory for all the pointers and tag_item structs.
The following results were measured with massif. Started MPD on
amd64, typed "mpc", no song being played. My music database contains
35k tagged songs. The results are what massif reports as "peak".
0.13.2: total 14,131,392; useful 11,408,972; extra 2,722,420
eric: total 18,370,696; useful 15,648,182; extra 2,722,514
mk f34f694: total 15,833,952; useful 13,111,470; extra 2,722,482
mk now: total 12,837,632; useful 10,626,383; extra 2,211,249
This patch set saves 20% memory, and does a good job in reducing heap
fragmentation.
The value is stored in the same memory allocation as the tag_item
struct; this saves memory because we do not store the value pointer
anymore. Also remove the getTagItemString()/removeTagItemString()
dummies.
Try to only include headers which are really needed. We should
particularly check all "headers including other headers". The
long-term goal is to have a manageable, small API for plugins
(decoders, output) without so many mpd internals cluttering the
namespace.
git-svn-id: https://svn.musicpd.org/mpd/trunk@7319 09075e82-0dd4-0310-85a5-a0d7c8717e4f
When dealing with in-memory lengths, the standard type "size_t" should
be used. Missing one can be quite dangerous, because an attacker
could provoke an integer under-/overflow, which may provide an attack
vector.
git-svn-id: https://svn.musicpd.org/mpd/trunk@7205 09075e82-0dd4-0310-85a5-a0d7c8717e4f
Local variables which are never read before the first assignment don't
need initialization. Saves a few bytes of text. Also don't reset
variables which are never read until function return.
git-svn-id: https://svn.musicpd.org/mpd/trunk@7199 09075e82-0dd4-0310-85a5-a0d7c8717e4f
[ew: cleaned up the dirty union hack a bit]
Signed-off-by: Eric Wong <normalperson@yhbt.net>
git-svn-id: https://svn.musicpd.org/mpd/trunk@7180 09075e82-0dd4-0310-85a5-a0d7c8717e4f
This will make refactoring features easier, especially now that
pthreads support and larger refactorings are on the horizon.
Hopefully, this will make porting to other platforms (even
non-UNIX-like ones for masochists) easier, too.
os_compat.h will house all the #includes for system headers
considered to be the "core" of MPD. Headers for optional
features will be left to individual source files.
git-svn-id: https://svn.musicpd.org/mpd/trunk@7130 09075e82-0dd4-0310-85a5-a0d7c8717e4f
thread-safety work in preparation for rewrite to use pthreads
Expect no regressions against trunk (r7078), possibly minor
performance improvements in update (due to fewer heap
allocations), but increased stack usage.
Applied the following patches:
* maxpath_str for reentrancy (temporary fix, reverted)
* path: start working on thread-safe variants of these methods
* Re-entrancy work on path/character-set conversions
* directory.c: exploreDirectory() use reentrant functions here
* directory/update: more use of reentrant functions + cleanups
* string_toupper: a strdup-less version of strDupToUpper
* get_song_url: a static-variable-free version of getSongUrl()
* Use reentrant/thread-safe get_song_url everywhere
* replace rmp2amp with the reentrant version, rmp2amp_r
* Get rid of the non-reentrant/non-thread-safe rpp2app, too.
* buffer2array: assert strdup() returns a usable value in unit tests
* replace utf8ToFsCharset and fsCharsetToUtf8 with thread-safe variants
* fix storing playlists w/o absolute paths
* parent_path(), a reentrant version of parentPath()
* parentPath => parent_path for reentrancy and thread-safety
* allow "make test" to automatically run embedded unit tests
* remove convStrDup() and maxpath_str()
* use MPD_PATH_MAX everywhere instead of MAXPATHLEN
* path: get rid of appendSlash, pfx_path and just use pfx_dir
* get_song_url: fix the ability to play songs in the top-level music_directory
git-svn-id: https://svn.musicpd.org/mpd/trunk@7106 09075e82-0dd4-0310-85a5-a0d7c8717e4f
and with different field types.
This fixes comments for id3v1 and id3v2
git-svn-id: https://svn.musicpd.org/mpd/trunk@7040 09075e82-0dd4-0310-85a5-a0d7c8717e4f
I'm checking for zero-size allocations and assert()-ing them,
so we can more easily get backtraces and debug problems, but we'll
also allow -DNDEBUG people to live on the edge if they wish.
We do not rely on errno when checking for OOM errors because
some implementations of malloc do not set it, and malloc
is commonly overridden by userspace wrappers.
I've spent some time looking through the source and didn't find any
obvious places where we would explicitly allocate 0 bytes, so we
shouldn't trip any of those assertions.
We also avoid allocating zero bytes because C libraries don't
handle this consistently (some return NULL, some not); and it's
dangerous either way.
git-svn-id: https://svn.musicpd.org/mpd/trunk@4690 09075e82-0dd4-0310-85a5-a0d7c8717e4f
