asn1: X.681/682/683 magic handling of open types

Status:

 - And it works!

 - We have an extensive test based on decoding a rich EK certficate.

   This test exercises all of:

    - decoding
    - encoding with and without decoded open types
    - copying of decoded values with decoded open types
    - freeing of decoded values with decoded open types

   Valgrind finds no memory errors.

 - Added a manual page for the compiler.

 - rfc2459.asn1 now has all three primary PKIX types that we care about
   defined as in RFC5912, with IOS constraints and parameterization:

    - `Extension`       (embeds open type in an `OCTET STRING`)
    - `OtherName`       (embeds open type in an        `ANY`-like type)
    - `SingleAttribute` (embeds open type in an        `ANY`-like type)
    - `AttributeSet`    (embeds open type in a  `SET OF ANY`-like type)

   All of these use OIDs as the open type type ID field, but integer
   open type type ID fields are also supported (and needed, for
   Kerberos).

   That will cover every typed hole pattern in all our ASN.1 modules.

   With this we'll be able to automatically and recursively decode
   through all subject DN attributes even when the subject DN is a
   directoryName SAN, and subjectDirectoryAttributes, and all
   extensions, and all SANs, and all authorization-data elements, and
   PA-data, and...

   We're not really using `SingleAttribute` and `AttributeSet` yet
   because various changes are needed in `lib/hx509` for that.

 - `asn1_compile` builds and recognizes the subset of X.681/682/683 that
   we need for, and now use in, rfc2459.asn1.  It builds the necessary
   AST, generates the correct C types, and generates templating for
   object sets and open types!

 - See READMEs for details.

 - Codegen backend not tested; I won't make it implement automatic open
   type handling, but it should at least not crash by substituting
   `heim_any` for open types not embedded in `OCTET STRING`.

 - We're _really_ starting to have problems with the ITU-T ASN.1
   grammar and our version of it...

   Type names have to start with upper-case, value names with
   lower-case, but it's not enough to disambiguate.

   The fact the we've allowed value and type names to violate their
   respective start-with case rules is causing us trouble now that we're
   adding grammar from X.681/682/683, and we're going to have to undo
   that.

   In preparation for that I'm capitalizing the `heim_any` and
   `heim_any_set` types, and doing some additional cleanup, which
   requires changes to other parts of Heimdal (all in this same commit
   for now).

   Problems we have because of this:

    - We cannot IMPORT values into modules because we have no idea if a
      symbol being imported refers to a value or a type because the only
      clue we would have is the symbol's name, so we assume IMPORTed
      symbols are for types.

      This means we can't import OIDs, for example, which is super
      annoying.

      One thing we might be able to do here is mark imported symbols as
      being of an undetermined-but-not-undefined type, then coerce the
      symbol's type the first time it's used in a context where its type
      is inferred as type, value, object, object set, or class.  (Though
      since we don't generate C symbols for objects or classes, we won't
      be able to import them, especially since we need to know them at
      compile time and cannot defer their handling to link- or
      run-time.)

    - The `NULL` type name, and the `NULL` value name now cause two
      reduce/reduce conflicts via the `FieldSetting` production.

    - Various shift/reduce conflicts involving `NULL` values in
      non-top-level contexts (in constraints, for example).

 - Currently I have a bug where to disambiguate the grammar I have a
   CLASS_IDENTIFIER token that is all caps, while TYPE_IDENTIFIER must
   start with a capital but not be all caps, but this breaks Kerberos
   since all its types are all capitalized -- oof!

   To fix this I made it so class names have to be all caps and
   start with an underscore (ick).

TBD:

 - Check all the XXX comments and address them
 - Apply this treatment to Kerberos!  Automatic handling of authz-data
   sounds useful :)
 - Apply this treatment to PKCS#10 (CSRs) and other ASN.1 modules too.
 - Replace various bits of code in `lib/hx509/` with uses of this
   feature.
 - Add JER.
 - Enhance `hxtool` and `asn1_print`.

Getting there!

lib/asn1/asn1-template.h

@@ -38,9 +38,55 @@
 #ifndef __TEMPLATE_H__
 #define __TEMPLATE_H__
 
+/*
+ * TBD:
+ * 
+ *  - For OER also encode number of optional/default/extension elements into
+ *    header entry's ptr field, not just the number of entries that follow it
+ *
+ *  - For JER/GSER/whatver, and probably for not-C-coded template interpreters,
+ *    we'll need to have an entry type for the names of structures and their
+ *    fields.
+ *
+ *  - For auto open types we need a new opcode, let's call it
+ *    A1_OP_OPENTYPE_OBJSET, and we need to encode into its entry:
+ *    a) the index of the template entry for the type ID field, and
+ *    b) the index of the template entry for the open type field,
+ *    c) 1 bit to indicate whether the object set is sorted by type ID value,
+ *    d) a pointer to the object set's template.
+ *    With that we can then find the struct offsets of those, and also their
+ *    types (since we can find their template entries).
+ *    The object set entries should be encoded into two template entries each:
+ *    one pointing to the value of the type ID field for that object (unless
+ *    the value is an integer, in which case the ptr should be the integer
+ *    value directly), and the other pointing to the template for the type
+ *    identified by the type ID.  These will need an opcode each...
+ *    A1_OP_OPENTYPE_ID and A1_OP_OPENTYPE.
+ *    We should also end the object set with an A1_OP_OPENTYPE_OBJSET entry so
+ *    that iterating backwards can be fast.  Unless... unless we don't inline
+ *    the object set and its objects but point to the object set's template.
+ *    Also, for extensible object sets we can point to the object set's name,
+ *    and we can then have a function to get an object set template by name,
+ *    one to release that, and one to add an object to the object set (there's
+ *    no need to remove objects from object sets, which helps with thread-
+ *    safety).  And then we don't need (c) either.
+ *    The decoder will then not see these entries until after decoding the type
+ *    ID and open type field (as its outer type, so OCTET STRING, BIT STRING,
+ *    or HEIM_ANY) and so it will be able to find those values in the struct at
+ *    their respective offsets.
+ *    The encoder and decoder both need to identify the relevant object in the
+ *    object set, either by linear search or binary search if they are sorted
+ *    by type ID value, then interpret the template for the identified type.
+ *    The encoder needs to place the encoding into the normal location for it
+ *    in the struct, then it can execute the normal template entry for it.
+ */
+
 /* header:
  *   HF  flags if not a BIT STRING type
  *   HBF flags if     a BIT STRING type
+ *
+ * ptr is count of elements
+ * offset is size of struct
  */
 
 /* tag:
@@ -49,6 +95,9 @@
  * 22..23 class
  * 24..27 flags
  * 28..31 op
+ *
+ * ptr points to template for tagged type
+ * offset is offset of struct field
  */
 
 /* parse:
@@ -56,11 +105,55 @@
  * 12..23 unused
  * 24..27 flags
  * 28..31 op
+ *
+ * ptr is NULL
+ * offset is ...
  */
 
 /* defval: (next template entry is defaulted)
  *
  *  DV    flags (ptr is or points to defval)
+ *
+ * ptr is default value or pointer to default value
+ * offset is all ones
+ */
+
+/* name: when it happens at index 1 it's the name of the SET/SEQUENCE/CHOICE
+ *       when it happens at any other index it's the name of the field that the
+ *       next entry deals with
+ *
+ *  0..23 unused
+ * 24..27 flags A1_NM_*
+ * 28..31 op
+ *
+ * ptr is const char * pointer to the name as C string
+ * offset is all zeros
+ */
+
+/* objset:
+ *  0..9  open type ID entry index
+ * 10..19 open type entry index
+ * 20..23 unused
+ * 24..27 flags A1_OS_*
+ * 28..31 op
+ *
+ * ptr points to object set template
+ * offset is the offset of the choice struct
+ */
+
+/* opentypeid: offset is zero
+ *             ptr points to value if it is not an integer
+ *             ptr   is the  value if it is     an integer
+ *  0..23 unused
+ * 24..27 flags A1_OTI_*
+ * 28..31 op
+ */
+
+/* opentype: offset is sizeof C type for this open type choice
+ *           ptr points to template for type choice
+ *  0..23 unused
+ * 24..27 flags
+ * 28..31 op
  */
 
 #define A1_OP_MASK		(0xf0000000)
@@ -73,6 +166,10 @@
 #define A1_OP_BMEMBER		(0x70000000)
 #define A1_OP_CHOICE		(0x80000000)
 #define A1_OP_DEFVAL		(0x90000000)
+#define A1_OP_OPENTYPE_OBJSET	(0xa0000000)
+#define A1_OP_OPENTYPE_ID	(0xb0000000)
+#define A1_OP_OPENTYPE		(0xc0000000)
+#define A1_OP_NAME		(0xd0000000)
 
 #define A1_FLAG_MASK		(0x0f000000)
 #define A1_FLAG_OPTIONAL	(0x01000000)
@@ -105,6 +202,10 @@
 #define A1_DV_INTEGER64		0x08
 #define A1_DV_UTF8STRING	0x10
 
+#define A1_OS_IS_SORTED		(0x01000000)
+#define A1_OS_OT_IS_ARRAY	(0x02000000)
+#define A1_OTI_IS_INTEGER	(0x04000000)
+
 
 struct asn1_template {
     uint32_t tt;