This adds a new backend for libhcrypto: the OpenSSL backend.
Now libhcrypto has these backends:
- hcrypto itself (i.e., the algorithms coded in lib/hcrypto)
- Common Crypto (OS X)
- PKCS#11 (specifically for Solaris, but not Solaris-specific)
- Windows CNG (Windows)
- OpenSSL (generic)
The ./configure --with-openssl=... option no longer disables the use of
hcrypto. Instead it enables the use of OpenSSL as a (and the default)
backend in libhcrypto. The libhcrypto framework is now always used.
OpenSSL should no longer be used directly within Heimdal, except in the
OpenSSL hcrypto backend itself, and files where elliptic curve (EC)
crypto is needed.
Because libhcrypto's EC support is incomplete, we can only use OpenSSL
for EC. Currently that means separating all EC-using code so that it
does not use hcrypto, thus the libhx509/hxtool and PKINIT EC code has
been moved out of the files it used to be in.
Normally one would dlopen() a shared object's basename, not its absolute
path. However, lib/krb5/plugin.c, in an effort to be zero-conf-ish,
wants to readdir() to find plugins to load, and in the process it ends
up defeating the RTLD's search-the-caller's-rpath.
This commit partially addresses this by allowing the use of $ORIGIN in
plugin_dir values and using them for the default (except on OS X).
This allows multiple Heimdal versions installed on the same host, but
with different plugin ABIs, to co-exist. A step forward for doing make
check on hosts where Heimdal is installed.
For now we hardcode $ORIGIN/../lib/plugin/krb5 (linux, Solaris, *BSD),
or $ORIGIN (Windows; for assemblies objects need to be in the same
directory) and we eval $ORIGIN by using dladdr() (Linux, Solaris) or
GetModuleHandleEx() (Win32, via a dladdr() wrapper in libroken) to find
the path to libkrb5 whose dirname to use as $ORIGIN. For Windows,
because we need the plugins to be in the same directory as libkrb5, we
require a prefix on plugin DLLs ("plugin_krb5_") to distinguish them
from other objects.
We should add a special token to mean "look in $ORIGIN, sure, but
dlopen() the plugin basenames only (so the RTLD can search the rpath)".
A colon can't be used to separate paths on Windows since they are used
in drive sepecification. Define a macro that can be used as a path
separator string. On Windows, this is defined as ";". It is a ":"
everywhere else.
HAVE_WINDNS is defined when we want to use DNSAPI on Windows which
handles most of the resolver work for us.
NO_LIMIT_FD_SETSIZE is defined if the value of a socket FD does not
imply anything about the number of sockets so far assigned.
I.e. checking if the socket is larger than FD_SETSIZE is incorrect.
