asn1: Allow CHOICEs to be decorated too

Prior to this commit only those C structs for SET and SEQUENCE types
could be decorated.  Now those for CHOICE types also can be decorated.

We could further extend this to SET OF and SEQUENCE OF types if it
proves useful.

lib/asn1/README.md

@@ -184,9 +184,42 @@ In recent times the following features have been added:
  - Most of X.690 is supported for decoding, with only DER supported for
    encoding.
 
- - Dumping of ASN.1 modules as JSON (note: the JSON schema for ASN.1 modules is
+ - For cryptographic applications there is a `--preserve-binary=TYPE` compiler
-   not stable yet).  This enables analysis using, e.g., `jq` or similar, and
+   option that causes the `TYPE`'s C `struct` to gain a `_save` field where the
-   even construction of alternative compilers using `jq` or similar.
+   original encoding of the `TYPE` is preserved by the decoder.  This allows
+   cryptographic applications to validate signatures, MACs, authenticated
+   decryption tags, checksums, etc., without having to re-encode the `TYPE`
+   (which wouldn't even work if the encoding received were BER and BER were
+   permitted for that `TYPE`).
+
+ - Unconstrained integer types have a large integer representation in C that is
+   not terribly useful in common cases.  Range and member constraints on
+   integer types cause the compiler to use `int`, `int64_t`, `unsigned int`,
+   and/or `uint64_t` as appropriate.
+
+ - The Heimdal ASN.1 compiler currently handles a large subset of X.680, and
+   (in a branch) a small subset of X.681, X.682, and X.683, which manifests as
+   automatic handling of all open types contained in `SET`/`SEQUENCE` types
+   that are parameterized with information object sets.  This allows all open
+   types in PKIX certificates, for example, to get decoded automatically no
+   matter how deeply nested.  We use a TCG EK certificate that has eight
+   certificate extensions, including subject alternative names and subject
+   directory attributes where the attribute values are not string types, and
+   all of these things get decoded automatically.
+
+ - The template backend dedups templates to save space.  This is an O(N^2) kind
+   of feature that we need to make optional, but it works.  (When we implement
+   JER this will have the side-effect of printing the wrong type names in some
+   cases because two or more types have the same templates and get deduped.)
+
+ - There is an _experimental_ ASN.1 module -> JSON feature in the compiler.  It
+   currently dumps type and value definitions, but not class, or object set
+   definitions.  Even for types, it is not complete, and the JSON schema used
+   is subject to change *WITHOUT NOTICE*.
+
+   Perhaps eventually we can re-write the compiler as a C-coded ASN.1 -> JSON
+   stage followed by a jq-coded code and template generator state, which would
+   make it much easier to extend the compiler.
 
  - We have an `asn1_print` program that can decode DER from any exported types
    from any ASN.1 modules committed in Heimdal:
@@ -787,35 +820,6 @@ In recent times the following features have been added:
    slightly different names in the ASN.1 modules in Heimdal's source tree.
    We'll fix this eventually.)
 
- - Unconstrained integer types have a large integer representation in C that is
-   not terribly useful in common cases.  Range and member constraints on
-   integer types cause the compiler to use `int`, `int64_t`, `unsigned int`,
-   and/or `uint64_t` as appropriate.
-
- - The Heimdal ASN.1 compiler currently handles a large subset of X.680, and
-   (in a branch) a small subset of X.681, X.682, and X.683, which manifests as
-   automatic handling of all open types contained in `SET`/`SEQUENCE` types
-   that are parameterized with information object sets.  This allows all open
-   types in PKIX certificates, for example, to get decoded automatically no
-   matter how deeply nested.  We use a TCG EK certificate that has eight
-   certificate extensions, including subject alternative names and subject
-   directory attributes where the attribute values are not string types, and
-   all of these things get decoded automatically.
-
- - The template backend dedups templates to save space.  This is an O(N^2) kind
-   of feature that we need to make optional, but it works.  (When we implement
-   JER this will have the side-effect of printing the wrong type names in some
-   cases because two or more types have the same templates and get deduped.)
-
- - There is an _experimental_ ASN.1 module -> JSON feature in the compiler.  It
-   currently dumps type and value definitions, but not class, or object set
-   definitions.  Even for types, it is not complete, and the JSON schema used
-   is subject to change *WITHOUT NOTICE*.
-
-   Perhaps eventually we can re-write the compiler as a C-coded ASN.1 -> JSON
-   stage followed by a jq-coded code and template generator state, which would
-   make it much easier to extend the compiler.
-
 ...
 
 ## Limitations
@@ -894,19 +898,19 @@ DESCRIPTION
      asn1_compile compiles an ASN.1 module into C source code and header
      files.
 
-     A fairly large subset of ASN.1 as specified in X.680, and the ASN.1 In-
+     A fairly large subset of ASN.1 as specified in X.680, and the ASN.1 In‐
-     formation Object System as specified in X.681, X.682, and X.683 is sup-
+     formation Object System as specified in X.681, X.682, and X.683 is sup‐
      ported, with support for the Distinguished Encoding Rules (DER), partial
-     Basic Encoding Rules (BER) support, and experimental JSON support (encod-
+     Basic Encoding Rules (BER) support, and experimental JSON support (encod‐
      ing only at this time).
 
-     See the compiler's README files for details about the C code and inter-
+     See the compiler's README files for details about the C code and inter‐
      faces it generates.
 
      The Information Object System support includes automatic codec support
      for encoding and decoding through “open types” which are also known as
-     “typed holes”.  See RFC 5912 for examples of how to use the ASN.1 Infor-
+     “typed holes”.  See RFC 5912 for examples of how to use the ASN.1 Infor‐
-     mation Object System via X.681/X.682/X.683 annotations.  See the com-
+     mation Object System via X.681/X.682/X.683 annotations.  See the com‐
      piler's README files for more information on ASN.1 Information Object
      System support.
 
@@ -918,7 +922,7 @@ DESCRIPTION
 	   •	   enable saving of as-received encodings of specific types
 		   for the purpose of signature validation;
 	   •	   generate add/remove utility functions for array types;
-	   •	   decorate generated ‘struct’ types with fields that are nei-
+	   •	   decorate generated ‘struct’ types with fields that are nei‐
 		   ther encoded nor decoded;
      etc.
 
@@ -945,7 +949,7 @@ DESCRIPTION
 	   •	   The REAL type is not supported.
 	   •	   The EmbeddedPDV type is not supported.
 	   •	   The BMPString type is not supported.
-	   •	   The IA5String is not properly supported, as it's essen-
+	   •	   The IA5String is not properly supported, as it's essen‐
 		   tially treated as a UTF8String with a different tag.
 	   •	   All supported non-octet strings are treated as like the
 		   UTF8String type.
@@ -967,19 +971,19 @@ DESCRIPTION
 	     The codegen backend generates C code for all functions directly,
 	     with no template interpretation.
 
-	     The template backend scales better than the codegen backend be-
+	     The template backend scales better than the codegen backend be‐
-	     cause as we add support for more encoding rules and more opera-
+	     cause as we add support for more encoding rules and more opera‐
 	     tions (we may add value comparators) the templates stay mostly
 	     the same, thus scaling linearly with size of module.  Whereas the
 	     codegen backend scales linear with the product of module size and
 	     number of encoding rules supported.
 
      --prefix-enum
-	     This option should be removed because ENUMERATED types should al-
+	     This option should be removed because ENUMERATED types should al‐
 	     ways have their labels prefixed.
 
      --enum-prefix=PREFIX
-	     This option should be removed because ENUMERATED types should al-
+	     This option should be removed because ENUMERATED types should al‐
 	     ways have their labels prefixed.
 
      --encode-rfc1510-bit-string
@@ -1012,15 +1016,15 @@ DESCRIPTION
 	     be a ‘SET OF’ or ‘SEQUENCE OF’ type.
 
      --decorate=ASN1-TYPE:FIELD-ASN1-TYPE:fname[?]
-	     Add to the C struct generated for the given ASN.1 SET or SEQUENCE
+	     Add to the C struct generated for the given ASN.1 SET, SEQUENCE,
-	     type named ASN1-TYPE a “hidden” field named fname of the given
+	     or CHOICE type named ASN1-TYPE a “hidden” field named fname of
-	     ASN.1 type FIELD-ASN1-TYPE, but do not encode or decode it.  If
+	     the given ASN.1 type FIELD-ASN1-TYPE, but do not encode or decode
-	     the fname ends in a question mark, then treat the field as OP-
+	     it.  If the fname ends in a question mark, then treat the field
-	     TIONAL.
+	     as OPTIONAL.
 
 	     This is useful for adding fields to existing types that can be
-	     used for internal bookkeeping but which do not affect interoper-
+	     used for internal bookkeeping but which do not affect interoper‐
-	     ability because they are neither encoded nor decoded.  For exam-
+	     ability because they are neither encoded nor decoded.  For exam‐
 	     ple, one might decorate a request type with state needed during
 	     processing of the request.
 
@@ -1028,31 +1032,31 @@ DESCRIPTION
 	     Add to the C struct generated for the given ASN.1 SET or SEQUENCE
 	     type named ASN1-TYPE a “hidden” field named fname of C type
 	     ‘heim_object_t’ values of which will be copied and released with
-	     ‘heim_release()’ and ‘heim_retain()’ respectively.
+	     ‘heim_retain()’ and ‘heim_release()’ respectively.
 
      --decorate=ASN1-TYPE:void*:fname
-	     Add to the C struct generated for the given ASN.1 SET or SEQUENCE
+	     Add to the C struct generated for the given ASN.1 SET, SEQUENCE,
-	     type named ASN1-TYPE a “hidden” field named fname of type ‘void
+	     or CHOICE type named ASN1-TYPE a “hidden” field named fname of
-	     *’ (but do not encode or decode it.
+	     type ‘void *’ (but do not encode or decode it.
 
 	     The destructor and copy constructor functions generated by this
 	     compiler for ASN1-TYPE will set this field to the ‘NULL’ pointer.
 
      --decorate=ASN1-TYPE:FIELD-C-TYPE:fname[?]:[copyfn]:[freefn]:header
-	     Add to the C struct generated for the given ASN.1 SET or SEQUENCE
+	     Add to the C struct generated for the given ASN.1 SET, SEQUENCE,
-	     type named ASN1-TYPE a “hidden” field named fname of the given
+	     or CHOICE type named ASN1-TYPE a “hidden” field named fname of
-	     external C type FIELD-C-TYPE, declared in the given header but do
+	     the given external C type FIELD-C-TYPE, declared in the given
-	     not encode or decode this field.  If the fname ends in a question
+	     header but do not encode or decode this field.  If the fname ends
-	     mark, then treat the field as OPTIONAL.
+	     in a question mark, then treat the field as OPTIONAL.
 
 	     The header must include double quotes or angle brackets.  The
 	     copyfn must be the name of a copy constructor function that takes
-	     a pointer to a source value of the type, and a pointer to a des-
+	     a pointer to a source value of the type, and a pointer to a des‐
 	     tination value of the type, in that order, and which returns zero
 	     on success or else a system error code on failure.	 The freefn
 	     must be the name of a destructor function that takes a pointer to
 	     a value of the type and which releases resources referenced by
-	     that value, but does not free the value itself (the run-time al-
+	     that value, but does not free the value itself (the run-time al‐
 	     locates this value as needed from the C heap).  The freefn should
 	     also reset the value to a pristine state (such as all zeros).
 
@@ -1060,8 +1064,6 @@ DESCRIPTION
 	     field will neither be copied nor freed by the functions generated
 	     for the TYPE.
 
-	     NOTE: At this time only one decoration may be specified per type.
-
      --one-code-file
 	     Generate a single source code file.  Otherwise a separate code
 	     file will be generated for every type.
@@ -1074,7 +1076,7 @@ DESCRIPTION
 
      --original-order
 	     Attempt to preserve the original order of type definition in the
-	     ASN.1 module.  By default the compiler generates types in a topo-
+	     ASN.1 module.  By default the compiler generates types in a topo‐
 	     logical sort order.
 
      --no-parse-units
@@ -1090,7 +1092,7 @@ DESCRIPTION
      --help
 
 NOTES
-     Currently only the template backend supports automatic encoding and de-
+     Currently only the template backend supports automatic encoding and de‐
      coding of open types via the ASN.1 Information Object System and
      X.681/X.682/X.683 annotations.
 

lib/asn1/asn1_compile.1

@@ -239,8 +239,8 @@ or
 .Sq SEQUENCE OF
 type.
 .It Fl Fl decorate=ASN1-TYPE:FIELD-ASN1-TYPE:fname[?]
-Add to the C struct generated for the given ASN.1 SET or SEQUENCE type
+Add to the C struct generated for the given ASN.1 SET, SEQUENCE, or
-named
+CHOICE type named
 .Ar ASN1-TYPE
 a
 .Dq hidden
@@ -274,8 +274,8 @@ and
 .Sq heim_release()
 respectively.
 .It Fl Fl decorate=ASN1-TYPE:void*:fname
-Add to the C struct generated for the given ASN.1 SET or SEQUENCE type
+Add to the C struct generated for the given ASN.1 SET, SEQUENCE, or
-named
+CHOICE type named
 .Ar ASN1-TYPE
 a
 .Dq hidden
@@ -292,8 +292,8 @@ will set this field to the
 .Sq NULL
 pointer.
 .It Fl Fl decorate=ASN1-TYPE:FIELD-C-TYPE:fname[?]:[copyfn]:[freefn]:header
-Add to the C struct generated for the given ASN.1 SET or SEQUENCE type
+Add to the C struct generated for the given ASN.1 SET, SEQUENCE, or
-named
+CHOICE type named
 .Ar ASN1-TYPE
 a
 .Dq hidden
@@ -335,8 +335,6 @@ and
 are empty strings, then the decoration field will neither be
 copied nor freed by the functions generated for the
 .Ar TYPE .
-.Pp
-NOTE: At this time only one decoration may be specified per type.
 .It Fl Fl one-code-file
 Generate a single source code file.
 Otherwise a separate code file will be generated for every type.

lib/asn1/check-gen.c

@@ -1047,6 +1047,9 @@ test_decorated(void)
     memset(&td, 0, sizeof(td));
     memset(&tnd, 0, sizeof(tnd));
 
+    my_copy_vers_called = 0;
+    my_free_vers_called = 0;
+
     td.version = 3;
     td.version3.v = 5;
     td.privthing = &td;
@@ -1085,7 +1088,8 @@ test_decorated(void)
         warnx("copy_TESTDecorated() did not work correctly (2)");
         return 1;
     }
-    if (td.version3.v != td_copy.version3.v) {
+    if (td.version3.v != td_copy.version3.v ||
+        my_copy_vers_called != 1) {
         warnx("copy_TESTDecorated() did not work correctly (3)");
         return 1;
     }
@@ -1119,6 +1123,97 @@ test_decorated(void)
     return 0;
 }
 
+static int
+test_decorated_choice(void)
+{
+    TESTNotDecoratedChoice tndc;
+    TESTDecoratedChoice tdc, tdc_copy;
+    size_t len, size;
+    void *ptr;
+    int ret;
+
+    memset(&tdc, 0, sizeof(tdc));
+    memset(&tndc, 0, sizeof(tndc));
+
+    my_copy_vers_called = 0;
+    my_free_vers_called = 0;
+
+    tdc.element = choice_TESTDecoratedChoice_version;
+    tdc.u.version = 3;
+    tdc.version3.v = 5;
+    tdc.privthing = &tdc;
+    tdc.privobj = heim_string_create("foo");
+    if ((tdc.version2 = malloc(sizeof(*tdc.version2))) == NULL)
+        errx(1, "out of memory");
+    *tdc.version2 = 5;
+    ASN1_MALLOC_ENCODE(TESTDecoratedChoice, ptr, len, &tdc, &size, ret);
+    if (ret) {
+        warnx("could not encode a TESTDecoratedChoice struct");
+        return 1;
+    }
+    ret = decode_TESTNotDecoratedChoice(ptr, len, &tndc, &size);
+    if (ret) {
+        warnx("could not decode a TESTDecoratedChoice struct as TESTNotDecoratedChoice");
+        return 1;
+    }
+    free(ptr);
+    if (size != len) {
+        warnx("TESTDecoratedChoice encoded size mismatch");
+        return 1;
+    }
+    if ((int)tdc.element != (int)tndc.element ||
+        tdc.u.version != tndc.u.version) {
+        warnx("TESTDecoratedChoice did not decode as a TESTNotDecoratedChoice correctly");
+        return 1;
+    }
+    if (copy_TESTDecoratedChoice(&tdc, &tdc_copy)) {
+        warnx("copy_TESTDecoratedChoice() failed");
+        return 1;
+    }
+    if ((int)tdc.element != (int)tdc_copy.element ||
+        tdc.u.version != tdc_copy.u.version) {
+        warnx("copy_TESTDecoratedChoice() did not work correctly (1)");
+        return 1;
+    }
+    if (tdc_copy.version2 == NULL || *tdc.version2 != *tdc_copy.version2) {
+        warnx("copy_TESTDecoratedChoice() did not work correctly (2)");
+        return 1;
+    }
+    if (tdc.version3.v != tdc_copy.version3.v ||
+        my_copy_vers_called != 1) {
+        warnx("copy_TESTDecoratedChoice() did not work correctly (3)");
+        return 1;
+    }
+    if (tdc_copy.privthing != 0) {
+        warnx("copy_TESTDecoratedChoice() did not work correctly (4)");
+        return 1;
+    }
+    if (tdc_copy.privobj != tdc.privobj) {
+        warnx("copy_TESTDecoratedChoice() did not work correctly (5)");
+        return 1;
+    }
+
+    free_TESTDecoratedChoice(&tdc_copy);
+    free_TESTDecoratedChoice(&tdc);
+    if (tdc.version2) {
+        warnx("free_TESTDecoratedChoice() did not work correctly (1)");
+        return 1;
+    }
+    if (tdc.version3.v != 0 || my_free_vers_called != 2) {
+        warnx("free_TESTDecoratedChoice() did not work correctly (2)");
+        return 1;
+    }
+    if (tdc.privthing != 0) {
+        warnx("free_TESTDecoratedChoice() did not work correctly (3)");
+        return 1;
+    }
+    if (tdc.privobj != 0) {
+        warnx("free_TESTDecoratedChoice() did not work correctly (4)");
+        return 1;
+    }
+    return 0;
+}
+
 
 static int
 cmp_TESTImplicit (void *a, void *b)
@@ -2597,6 +2692,7 @@ main(int argc, char **argv)
 
     DO_ONE(test_default);
 
+    DO_ONE(test_decorated_choice);
     DO_ONE(test_decorated);
 
 #if ASN1_IOS_SUPPORTED

lib/asn1/gen.c

@@ -1488,6 +1488,9 @@ define_type(int level, const char *name, const char *basename, Type *pt, Type *t
         define_type(level, name, basename, t, t->subtype, typedefp, preservep);
 	break;
     case TChoice: {
+        struct decoration deco;
+        ssize_t more_deco = -1;
+        int decorated = 0;
 	int first = 1;
 	Member *m;
 
@@ -1546,9 +1549,39 @@ define_type(int level, const char *name, const char *basename, Type *pt, Type *t
 	}
 	space(level + 1);
 	fprintf (headerfile, "} u;\n");
+        fprintf(jsonfile, "]");
+
+        while (decorate_type(newbasename, &deco, &more_deco)) {
+            decorated++;
+	    space(level + 1);
+            fprintf(headerfile, "%s %s%s;\n", deco.field_type,
+                    deco.opt ? "*" : "", deco.field_name);
+            if (deco.first)
+                fprintf(jsonfile, ",\"decorate\":[");
+            fprintf(jsonfile, "%s{"
+                    "\"type\":\"%s\",\"name\":\"%s\",\"optional\":%s,"
+                    "\"external\":%s,\"pointer\":%s,\"void_star\":%s,"
+                    "\"struct_star\":%s,\"heim_object\":%s,"
+                    "\"copy_function\":\"%s\","
+                    "\"free_function\":\"%s\",\"header_name\":%s%s%s"
+                    "}",
+                    deco.first ? "" : ",",
+                    deco.field_type, deco.field_name,
+                    deco.opt ? "true" : "false", deco.ext ? "true" : "false",
+                    deco.ptr ? "true" : "false", deco.void_star ? "true" : "false",
+                    deco.struct_star ? "true" : "false", deco.heim_object ? "true" : "false",
+                    deco.copy_function_name ? deco.copy_function_name : "",
+                    deco.free_function_name ? deco.free_function_name : "",
+                    deco.header_name && deco.header_name[0] == '"' ? "" : "\"",
+                    deco.header_name ? deco.header_name : "",
+                    deco.header_name && deco.header_name[0] == '"' ? "" : "\""
+                    );
+        }
+        if (decorated)
+            fprintf(jsonfile, "]");
+
 	space(level);
 	fprintf (headerfile, "} %s;\n", name);
-        fprintf(jsonfile, "]");
 	break;
     }
     case TUTCTime:
@@ -1669,10 +1702,19 @@ declare_type(const Symbol *s, Type *t, int typedefp)
 	getnewbasename(&newbasename, TRUE, s->gen_name, s->gen_name);
 	fprintf(headerfile, "struct %s %s;\n", newbasename, s->gen_name);
 	break;
-    case TChoice:
+    case TChoice: {
+        struct decoration deco;
+        ssize_t more_deco = -1;
+
 	getnewbasename(&newbasename, TRUE, s->gen_name, s->gen_name);
 	fprintf(headerfile, "struct %s %s;\n", newbasename, s->gen_name);
+        while (decorate_type(newbasename, &deco, &more_deco)) {
+            if (deco.header_name)
+                fprintf(headerfile, "#include %s\n", deco.header_name);
+            free(deco.field_type);
+        }
 	break;
+    }
     default:
 	abort ();
     }

lib/asn1/gen_template.c

@@ -1317,6 +1317,8 @@ template_members(struct templatehead *temp,
 	break;
     }
     case TChoice: {
+        struct decoration deco;
+        ssize_t more_deco = -1;
 	struct templatehead template;
 	struct template *q;
 	size_t count = 0, i;
@@ -1398,6 +1400,34 @@ template_members(struct templatehead *temp,
 
 	add_line(temp, "{ A1_OP_CHOICE, %s, %s }", poffset, tname);
 
+        while (decorate_type(basetype, &deco, &more_deco)) {
+            char *poffset2;
+
+            poffset2 = partial_offset(basetype, deco.field_name, 1, isstruct);
+
+            if (deco.ext && deco.heim_object) {
+                add_line_string(temp, "0", poffset2,
+                                "A1_OP_TYPE_DECORATE_EXTERN |A1_FLAG_HEIM_OBJ");
+            } else if (deco.ext) {
+                char *ptr = NULL;
+
+                /* Decorated with external C type */
+                if (asprintf(&ptr, "&asn1_extern_%s_%s",
+                             basetype, deco.field_name) == -1 || ptr == NULL)
+                    err(1, "out of memory");
+                add_line_pointer(temp, ptr, poffset2,
+                                 "A1_OP_TYPE_DECORATE_EXTERN %s",
+                                 deco.opt ? "|A1_FLAG_OPTIONAL" : "");
+                free(ptr);
+            } else
+                /* Decorated with a templated ASN.1 type */
+                add_line_pointer(temp, deco.field_type, poffset2,
+                                 "A1_OP_TYPE_DECORATE %s",
+                                 deco.opt ? "|A1_FLAG_OPTIONAL" : "");
+            free(poffset2);
+            free(deco.field_type);
+        }
+
 	free(e);
 	free(tname);
 	break;

lib/asn1/test.asn1

@@ -296,4 +296,14 @@ TESTNotDecorated ::= SEQUENCE {
   -- should have the same encoding as TESTDecorated
 }
 
+TESTDecoratedChoice ::= CHOICE {
+  version     TESTuint32
+  -- gets decorated with varius fields (see test.opt)
+}
+
+TESTNotDecoratedChoice ::= CHOICE {
+  version     TESTuint32
+  -- should have the same encoding as TESTDecoratedChoice
+}
+
 END

lib/asn1/test.opt

@@ -3,3 +3,7 @@
 --decorate=TESTDecorated:my_vers:version3:my_copy_vers:my_free_vers:"check-gen.h"
 --decorate=TESTDecorated:void *:privthing
 --decorate=TESTDecorated:heim_object_t:privobj
+--decorate=TESTDecoratedChoice:TESTuint32:version2?
+--decorate=TESTDecoratedChoice:my_vers:version3:my_copy_vers:my_free_vers:"check-gen.h"
+--decorate=TESTDecoratedChoice:void *:privthing
+--decorate=TESTDecoratedChoice:heim_object_t:privobj