This is the second of two commits in a series that must be picked together.
This series of two commits moves parts of lib/krb5/ infrastructure
functionality to lib/base/, leaving behind wrappers.
Some parts of libkrb5 are entirely generic or easily made so, and could
be useful in various parts of Heimdal that are not specific to the krb5
API, such as:
- lib/gssapi/ (especially since the integration of NegoEx)
- lib/hx509/
- bx509d (which should really move out of kdc/)
For the above we need to move these bits of lib/krb5/:
- lib/krb5/config_file.c (all of it, leaving forwardings behind)
- lib/krb5/config_reg.c (all of it)
- lib/krb5/plugin.c (all of it, leaving forwardings behind)
- lib/krb5/log.c (all of it, ditto)
- lib/krb5/heim_err.et (all of it)
And because of those two, these too must also move:
- lib/krb5/expand_path.c (all of it, leaving forwardings behind)
- lib/krb5/warn.c (just the warning functions, ditto)
The changes to the moved files are mostly quite straightforward and are
best reviewed with --word-diff=color.
We're also creating a heim_context and a heim API to go with it. But
it's as thin as possible, with as little state as necessary to enable
this move. Functions for dealing with error messages use callbacks.
Moving plugin.c does have one knock-on effect on all users of the old
krb5 plugin API (which remains), which is that a global search and
replace of struct krb5_plugin_data to struct heim_plugin_data was
needed, though the layout and size of that structure doesn't change, so
the ABI doesn't either.
As well, we now build lib/vers/ and lib/com_err/ before lib/base/ so as
to be able to move lib/krb5/heim_err.et to lib/base/ so that we can make
use of HEIM_ERR_* in lib/base/, specifically in the files that moved.
Once this is all done we'll be able to use config files and plugins in
lib/hx509/, we'll be able to move bx509d out of kdc/, and so on.
Most if not all of the new functions in lib/base/ are Heimdal-private,
thus calling conventions for them are not declared.
Status:
- builds and passes CIs (Travis, Appveyor)
- ran make check-valgrind and no new leaks or other memory errors
- ready for review
HOW TO REVIEW:
$ # Review file moves:
$ git log --stat -n1 HEAD^
$
$ # Review changes to moved files using --word-diff=color
$ git log -p -b -w --word-diff=color HEAD^..HEAD \
lib/base/config_file.c \
lib/base/config_reg.c \
lib/base/expand_path.c \
lib/base/warn.c \
lib/krb5/config_file.c \
lib/krb5/config_reg.c \
lib/krb5/expand_path.c \
lib/krb5/warn.c
$
$ # Review the whole thing, possibly adding -b and/or -w, and
$ # maybe --word-diff=color:
$ git log -p origin/master..HEAD
$ git log -p -b -w origin/master..HEAD
$ git log -p -b -w --word-diff=color origin/master..HEAD
TBD (future commits):
- make lib/gssapi use the new heimbase functions
- move kx509/bx509d common code to lib/hx509/ or other approp. location
- move bx509d out of kdc/
Refactor plugin framework to use a single list of loaded plugins; add a new
plugin API where DSOs export a load function that can declare dependencies and
export multiple plugins; refactor kadm5 hook API to use krb5 plugin framework.
More information in krb5-plugin(7).
[Code reviewed by Love Hörnquist Åstrand <lha@kth.se>]
Added heim_db_*() entry points for dealing with databases, and
make krb5_aname_to_localname() use it.
The following enhancements to libheimbase are included:
- Add heim_data_t and heim_string_t "reference" variants to
avoid memory copies of potentially large data/strings.
See heim_data_ref_create() and heim_string_ref_create().
- Added enhancements to heim_array_t to allow their use for
queues and stacks, and to improve performance. See
heim_array_insert_value().
- Added XPath-like accessors for heim_object_t. See
heim_path_get(), heim_path_copy(), heim_path_create(), and
heim_path_delete(). These are used extensively in the DB
framework's generic composition of ACID support and in the
test_base program
- Made libheimbase more consistent with Core Foundation naming
conventions. See heim_{dict, array}_{get, copy}_value() and
heim_path_{get, copy}().
- Added functionality to and fixed bugs in base/json.c:
- heim_serialize();
- depth limit for JSON parsing (for DoS protection);
- pretty-printing;
- JSON compliance (see below);
- flag options for parsing and serializing; these are needed
because of impedance mismatches between heim_object_t and
JSON (e.g., heim_dict_t allows non-string keys, but JSON
does not; heimbase supports binary data, while JSON does
not).
- Added heim_error_enomem().
- Enhanced the test_base program to test new functionality and
to use heim_path*() to better test JSON encoding. This
includes some fuzz testing of JSON parsing, and running the
test under valgrind.
- Started to add doxygen documentation for libheimbase (but doc
build for libheimbase is still incomplete).
Note that there's still some incomplete JSON support:
- JSON string quoting is not fully implemented;
- libheimbase lacks support for real numbers, while JSON has
it -- otherwise libheimbase is a superset of JSON,
specifically in that any heim_object_t can be a key for an
associative array.
The following DB backends are supported natively:
- "sorted-text", a binary search of sorted (in C locale), flat
text files;
- "json", a backend that stores DB contents serialized as JSON
(this is intended for configuration-like contents).
The DB framework supports:
- multiple key/value tables per-DB
- ACID transactions
The DB framework also natively implements ACID transactions for
any DB backends that a) do not provide transactions natively, b)
do provide lock/unlock/sync methods (even on Windows). This
includes autocommit of DB updates outside transactions.
Future DB enhancements may include:
- add backends for various DB types (BDB, CDB, MDB, ...);
- make libhdb use heim_db_t;
- add a command-line tool for interfacing to databases via
libheimbase (e.g., to get/set/delete values, create/copy/
backup DBs, inspect history, check integrity);
- framework-level transaction logging (with redo and undo
logging), for generic incremental replication;
- framework-level DB integrity checking.
We could store a MAC of the XOR of a hash function applied to
{key, value} for every entry in the DB, then use this to check
DB integrity incrementally during incremental replication, as
well as for the whole DB.
