oysteikt/heimdal
0
0
Fork 0
Code Issues 5 Pull Requests Releases Activity

asn1: X.681/682/683 magic handling of open types

Browse Source 
Status:

 - And it works!

 - We have an extensive test based on decoding a rich EK certficate.

   This test exercises all of:

    - decoding
    - encoding with and without decoded open types
    - copying of decoded values with decoded open types
    - freeing of decoded values with decoded open types

   Valgrind finds no memory errors.

 - Added a manual page for the compiler.

 - rfc2459.asn1 now has all three primary PKIX types that we care about
   defined as in RFC5912, with IOS constraints and parameterization:

    - `Extension`       (embeds open type in an `OCTET STRING`)
    - `OtherName`       (embeds open type in an        `ANY`-like type)
    - `SingleAttribute` (embeds open type in an        `ANY`-like type)
    - `AttributeSet`    (embeds open type in a  `SET OF ANY`-like type)

   All of these use OIDs as the open type type ID field, but integer
   open type type ID fields are also supported (and needed, for
   Kerberos).

   That will cover every typed hole pattern in all our ASN.1 modules.

   With this we'll be able to automatically and recursively decode
   through all subject DN attributes even when the subject DN is a
   directoryName SAN, and subjectDirectoryAttributes, and all
   extensions, and all SANs, and all authorization-data elements, and
   PA-data, and...

   We're not really using `SingleAttribute` and `AttributeSet` yet
   because various changes are needed in `lib/hx509` for that.

 - `asn1_compile` builds and recognizes the subset of X.681/682/683 that
   we need for, and now use in, rfc2459.asn1.  It builds the necessary
   AST, generates the correct C types, and generates templating for
   object sets and open types!

 - See READMEs for details.

 - Codegen backend not tested; I won't make it implement automatic open
   type handling, but it should at least not crash by substituting
   `heim_any` for open types not embedded in `OCTET STRING`.

 - We're _really_ starting to have problems with the ITU-T ASN.1
   grammar and our version of it...

   Type names have to start with upper-case, value names with
   lower-case, but it's not enough to disambiguate.

   The fact the we've allowed value and type names to violate their
   respective start-with case rules is causing us trouble now that we're
   adding grammar from X.681/682/683, and we're going to have to undo
   that.

   In preparation for that I'm capitalizing the `heim_any` and
   `heim_any_set` types, and doing some additional cleanup, which
   requires changes to other parts of Heimdal (all in this same commit
   for now).

   Problems we have because of this:

    - We cannot IMPORT values into modules because we have no idea if a
      symbol being imported refers to a value or a type because the only
      clue we would have is the symbol's name, so we assume IMPORTed
      symbols are for types.

      This means we can't import OIDs, for example, which is super
      annoying.

      One thing we might be able to do here is mark imported symbols as
      being of an undetermined-but-not-undefined type, then coerce the
      symbol's type the first time it's used in a context where its type
      is inferred as type, value, object, object set, or class.  (Though
      since we don't generate C symbols for objects or classes, we won't
      be able to import them, especially since we need to know them at
      compile time and cannot defer their handling to link- or
      run-time.)

    - The `NULL` type name, and the `NULL` value name now cause two
      reduce/reduce conflicts via the `FieldSetting` production.

    - Various shift/reduce conflicts involving `NULL` values in
      non-top-level contexts (in constraints, for example).

 - Currently I have a bug where to disambiguate the grammar I have a
   CLASS_IDENTIFIER token that is all caps, while TYPE_IDENTIFIER must
   start with a capital but not be all caps, but this breaks Kerberos
   since all its types are all capitalized -- oof!

   To fix this I made it so class names have to be all caps and
   start with an underscore (ick).

TBD:

 - Check all the XXX comments and address them
 - Apply this treatment to Kerberos!  Automatic handling of authz-data
   sounds useful :)
 - Apply this treatment to PKCS#10 (CSRs) and other ASN.1 modules too.
 - Replace various bits of code in `lib/hx509/` with uses of this
   feature.
 - Add JER.
 - Enhance `hxtool` and `asn1_print`.

Getting there!
This commit is contained in:
Nicolas Williams
2021-02-08 22:40:51 -06:00
parent 89f97e8287
commit db7763ca7b
64 changed files with 5076 additions and 850 deletions
Download Patch File Download Diff File Expand all files Collapse all files

280
lib/asn1/gen.c
View File

@@ -234,7 +234,9 @@ init_generate (const char *filename, const char *base)
"} heim_bit_string;\n\n");
fprintf (headerfile,
"typedef struct heim_base_data heim_any;\n"
"typedef struct heim_base_data heim_any_set;\n\n");
"typedef struct heim_base_data heim_any_set;\n"
"typedef struct heim_base_data HEIM_ANY;\n"
"typedef struct heim_base_data HEIM_ANY_SET;\n\n");
fputs("#define ASN1_MALLOC_ENCODE(T, B, BL, S, L, R) \\\n"
" do { \\\n"
" (BL) = length_##T((S)); \\\n"
@@ -643,7 +645,12 @@ define_asn1 (int level, Type *t)
{
switch (t->type) {
case TType:
fprintf (headerfile, "%s", t->symbol->name);
if (!t->symbol && t->typeref.iosclass) {
fprintf(headerfile, "%s.&%s",
t->typeref.iosclass->symbol->name,
t->typeref.field->name);
} else
fprintf(headerfile, "%s", t->symbol->name);
break;
case TInteger:
if(t->members == NULL) {
@@ -810,8 +817,215 @@ getnewbasename(char **newbasename, int typedefp, const char *basename, const cha
err(1, "malloc");
}
static void define_type(int, const char *, const char *, Type *, Type *, int, int);
/*
* Get the SET/SEQUENCE member pair and CLASS field pair defining an open type.
*
* There are three cases:
*
* - open types embedded in OCTET STRING, with the open type object class
* relation declared via a constraint
*
* - open types not embedded in OCTET STRING, which are really more like ANY
* DEFINED BY types, so, HEIM_ANY
*
* - open types in a nested structure member where the type ID field is in a
* member of the ancestor structure (this happens in PKIX's `AttributeSet',
* where the open type is essentially a SET OF HEIM_ANY).
*
* In a type like PKIX's SingleAttribute the type ID member would be the one
* named "type" and the open type member would be the one named "value", and
* the corresponding fields of the ATTRIBUTE class would be named "id" and
* "Type".
*
* NOTE: We assume a single open type member pair in any SET/SEQUENCE. In
* principle there could be more pairs and we could iterate them, or
* better yet, we could be given the name of an open type member and then
* just find its related type ID member and fields, then our caller would
* iterate the SET/SEQUENCE type's members looking for open type members
* and would call this function for each one found.
*/
void
get_open_type_defn_fields(const Type *t,
Member **typeidmember,
Member **opentypemember,
Field **typeidfield,
Field **opentypefield,
int *is_array_of)
{
Member *m;
Field *junk1, *junk2;
char *idmembername = NULL;
if (!typeidfield) typeidfield = &junk1;
if (!opentypefield) opentypefield = &junk2;
*typeidfield = *opentypefield = NULL;
*typeidmember = *opentypemember = NULL;
*is_array_of = 0;
/* Look for the open type member */
HEIM_TAILQ_FOREACH(m, t->members, members) {
Type *subtype = m->type;
Type *sOfType = NULL;
while (subtype->type == TTag ||
subtype->type == TSetOf ||
subtype->type == TSequenceOf) {
if (subtype->type == TTag && subtype->subtype) {
if (subtype->subtype->type == TOctetString ||
subtype->subtype->type == TBitString)
break;
subtype = subtype->subtype;
} else if (subtype->type == TSetOf || subtype->type == TSequenceOf) {
sOfType = subtype;
if (sOfType->symbol)
break;
if (subtype->subtype)
subtype = subtype->subtype;
} else
break;
}
/*
* If we traversed through a non-inlined SET OF or SEQUENCE OF type,
* then this cannot be an open type field.
*/
if (sOfType && sOfType->symbol)
continue;
/*
* The type of the field we're interested in has to have an information
* object constraint.
*/
if (!subtype->constraint)
continue;
if (subtype->type != TType && subtype->type != TTag)
continue;
/*
* Check if it's an ANY-like member or like an OCTET STRING CONTAINING
* member. Those are the only two possibilities.
*/
if ((subtype->type == TTag || subtype->type == TType) &&
subtype->subtype &&
subtype->constraint->ctype == CT_CONTENTS &&
subtype->constraint->u.content.type &&
subtype->constraint->u.content.type->type == TType &&
!subtype->constraint->u.content.type->subtype &&
subtype->constraint->u.content.type->constraint &&
subtype->constraint->u.content.type->constraint->ctype == CT_TABLE_CONSTRAINT) {
/* Type like OCTET STRING or BIT STRING CONTAINING open type */
*opentypemember = m;
*opentypefield = subtype->constraint->u.content.type->typeref.field;
*is_array_of = sOfType != NULL;
idmembername = subtype->constraint->u.content.type->constraint->u.content.crel.membername;
break;
} else if (subtype->symbol && strcmp(subtype->symbol->name, "HEIM_ANY") == 0) {
/* Open type, but NOT embedded in OCTET STRING or BIT STRING */
*opentypemember = m;
*opentypefield = subtype->typeref.field;
*is_array_of = sOfType != NULL;
idmembername = subtype->constraint->u.content.crel.membername;
break;
}
}
/* Look for the type ID member identified in the previous loop */
HEIM_TAILQ_FOREACH(m, t->members, members) {
if (!m->type->subtype || strcmp(m->name, idmembername))
continue;
if (m->type->constraint &&
m->type->constraint->ctype == CT_TABLE_CONSTRAINT)
*typeidfield = m->type->typeref.field;
else if (m->type->subtype->constraint &&
m->type->subtype->constraint->ctype == CT_TABLE_CONSTRAINT)
*typeidfield = m->type->subtype->typeref.field;
else
continue;
/* This is the type ID field (because there _is_ a subtype) */
*typeidmember = m;
break;
}
}
/*
* Generate CHOICE-like struct fields for open types declared via
* X.681/682/683 syntax.
*
* We could support multiple open type members in a SET/SEQUENCE, but for now
* we support only one.
*/
static void
define_type (int level, const char *name, const char *basename, Type *t, int typedefp, int preservep)
define_open_type(int level, const char *newbasename, const char *name, const char *basename, Type *pt, Type *t)
{
Member *opentypemember, *typeidmember;
Field *opentypefield, *typeidfield;
ObjectField *of;
IOSObjectSet *os = pt->actual_parameter;
IOSObject **objects;
size_t nobjs, i;
int is_array_of_open_type;
get_open_type_defn_fields(pt, &typeidmember, &opentypemember,
&typeidfield, &opentypefield,
&is_array_of_open_type);
if (!opentypemember || !typeidmember ||
!opentypefield || !typeidfield)
errx(1, "Open type specification in %s is incomplete", name);
sort_object_set(os, typeidfield, &objects, &nobjs);
fprintf(headerfile, "struct {\n");
/* Iterate objects in the object set, gen enum labels */
fprintf(headerfile, "enum { choice_%s_iosnumunknown = 0,\n",
newbasename);
for (i = 0; i < nobjs; i++) {
HEIM_TAILQ_FOREACH(of, objects[i]->objfields, objfields) {
if (strcmp(of->name, typeidfield->name))
continue;
if (!of->value || !of->value->s)
errx(1, "Unknown value in value field %s of object %s",
of->name, objects[i]->symbol->name);
fprintf(headerfile, "choice_%s_iosnum_%s,\n",
newbasename, of->value->s->gen_name);
}
}
fprintf(headerfile, "} element;\n");
if (is_array_of_open_type)
fprintf(headerfile, "unsigned int len;\n");
/* Iterate objects in the object set, gen union arms */
fprintf(headerfile, "union {\nvoid *_any;\n");
for (i = 0; i < nobjs; i++) {
HEIM_TAILQ_FOREACH(of, objects[i]->objfields, objfields) {
char *n = NULL;
if (strcmp(of->name, opentypefield->name))
continue;
if (!of->type || (!of->type->symbol && of->type->type != TTag) ||
of->type->tag.tagclass != ASN1_C_UNIV) {
warnx("Ignoring unknown or unset type field %s of object %s",
of->name, objects[i]->symbol->name);
continue;
}
if (asprintf(&n, "*%s", objects[i]->symbol->gen_name) < 0 || n == NULL)
err(1, "malloc");
define_type(level + 2, n, newbasename, NULL, of->type, FALSE, FALSE);
free(n);
}
}
if (is_array_of_open_type) {
fprintf(headerfile, "} *val;\n} _ioschoice_%s;\n", opentypemember->gen_name);
} else {
fprintf(headerfile, "} u;\n");
fprintf(headerfile, "} _ioschoice_%s;\n", opentypemember->gen_name);
}
free(objects);
}
static void
define_type(int level, const char *name, const char *basename, Type *pt, Type *t, int typedefp, int preservep)
{
const char *label_prefix = NULL;
const char *label_prefix_sep = NULL;
@@ -820,7 +1034,12 @@ define_type (int level, const char *name, const char *basename, Type *t, int typ
switch (t->type) {
case TType:
space(level);
fprintf(headerfile, "%s %s;\n", t->symbol->gen_name, name);
if (!t->symbol && t->actual_parameter)
define_open_type(level, newbasename, name, basename, t, t);
else if (!t->symbol && pt->actual_parameter)
define_open_type(level, newbasename, name, basename, pt, t);
else
fprintf(headerfile, "%s %s;\n", t->symbol->gen_name, name);
break;
case TInteger:
if (t->symbol && t->symbol->emitted_definition)
@@ -915,7 +1134,7 @@ define_type (int level, const char *name, const char *basename, Type *t, int typ
while (pos < m->val) {
if (asprintf (&n, "_unused%d:1", pos) < 0 || n == NULL)
err(1, "malloc");
define_type (level + 1, n, newbasename, &i, FALSE, FALSE);
define_type(level + 1, n, newbasename, NULL, &i, FALSE, FALSE);
free(n);
pos++;
}
@@ -923,7 +1142,7 @@ define_type (int level, const char *name, const char *basename, Type *t, int typ
n = NULL;
if (asprintf (&n, "%s:1", m->gen_name) < 0 || n == NULL)
errx(1, "malloc");
define_type (level + 1, n, newbasename, &i, FALSE, FALSE);
define_type(level + 1, n, newbasename, NULL, &i, FALSE, FALSE);
free (n);
n = NULL;
pos++;
@@ -938,7 +1157,7 @@ define_type (int level, const char *name, const char *basename, Type *t, int typ
char *n = NULL;
if (asprintf (&n, "_unused%d:1", pos) < 0 || n == NULL)
errx(1, "malloc");
define_type (level + 1, n, newbasename, &i, FALSE, FALSE);
define_type(level + 1, n, newbasename, NULL, &i, FALSE, FALSE);
free(n);
pos++;
}
@@ -989,13 +1208,15 @@ define_type (int level, const char *name, const char *basename, Type *t, int typ
} else if (m->optional) {
char *n = NULL;
if (asprintf (&n, "*%s", m->gen_name) < 0 || n == NULL)
if (asprintf(&n, "*%s", m->gen_name) < 0 || n == NULL)
errx(1, "malloc");
define_type (level + 1, n, newbasename, m->type, FALSE, FALSE);
define_type(level + 1, n, newbasename, t, m->type, FALSE, FALSE);
free (n);
} else
define_type (level + 1, m->gen_name, newbasename, m->type, FALSE, FALSE);
define_type(level + 1, m->gen_name, newbasename, t, m->type, FALSE, FALSE);
}
if (t->actual_parameter && t->actual_parameter->objects)
define_open_type(level, newbasename, name, basename, t, t);
space(level);
fprintf (headerfile, "} %s;\n", name);
break;
@@ -1013,8 +1234,8 @@ define_type (int level, const char *name, const char *basename, Type *t, int typ
space(level);
fprintf (headerfile, "struct %s {\n", newbasename);
define_type (level + 1, "len", newbasename, &i, FALSE, FALSE);
define_type (level + 1, "*val", newbasename, t->subtype, FALSE, FALSE);
define_type(level + 1, "len", newbasename, t, &i, FALSE, FALSE);
define_type(level + 1, "*val", newbasename, t, t->subtype, FALSE, FALSE);
space(level);
fprintf (headerfile, "} %s;\n", name);
break;
@@ -1032,7 +1253,7 @@ define_type (int level, const char *name, const char *basename, Type *t, int typ
fprintf (headerfile, "heim_general_string %s;\n", name);
break;
case TTag:
define_type (level, name, basename, t->subtype, typedefp, preservep);
define_type(level, name, basename, t, t->subtype, typedefp, preservep);
break;
case TChoice: {
int first = 1;
@@ -1077,10 +1298,10 @@ define_type (int level, const char *name, const char *basename, Type *t, int typ
if (asprintf (&n, "*%s", m->gen_name) < 0 || n == NULL)
errx(1, "malloc");
define_type (level + 2, n, newbasename, m->type, FALSE, FALSE);
define_type(level + 2, n, newbasename, t, m->type, FALSE, FALSE);
free (n);
} else
define_type (level + 2, m->gen_name, newbasename, m->type, FALSE, FALSE);
define_type(level + 2, m->gen_name, newbasename, t, m->type, FALSE, FALSE);
}
space(level + 1);
fprintf (headerfile, "} u;\n");
@@ -1140,7 +1361,7 @@ declare_type(const Symbol *s, Type *t, int typedefp)
switch (t->type) {
case TType:
define_type(0, s->gen_name, s->gen_name, s->type, TRUE, TRUE);
define_type(0, s->gen_name, s->gen_name, NULL, s->type, TRUE, TRUE);
if (template_flag)
generate_template_type_forward(s->gen_name);
emitted_declaration(s);
@@ -1162,7 +1383,7 @@ declare_type(const Symbol *s, Type *t, int typedefp)
case TVisibleString:
case TOID :
case TNull:
define_type(0, s->gen_name, s->gen_name, s->type, TRUE, TRUE);
define_type(0, s->gen_name, s->gen_name, NULL, s->type, TRUE, TRUE);
if (template_flag)
generate_template_type_forward(s->gen_name);
emitted_declaration(s);
@@ -1295,10 +1516,12 @@ generate_subtypes_header(const Symbol *s)
switch (t->type) {
default: return;
case TType:
if (t->symbol && (s = getsym(t->symbol->name)))
generate_type_header(s);
case TType: {
Symbol *s2;
if (t->symbol && (s2 = getsym(t->symbol->name)) != s)
generate_type_header(s2);
return;
}
case TSet:
case TSequence:
case TChoice:
@@ -1342,10 +1565,14 @@ generate_type_header (const Symbol *s)
* needed in stripped objects.
*/
if (!s->emitted_tag_enums) {
while (t->type == TType && s->type->symbol && s->type->symbol->type)
t = s->type->symbol->type;
while (t->type == TType && s->type->symbol && s->type->symbol->type) {
if (t->subtype)
t = t->subtype;
else
t = s->type->symbol->type;
}
if (t->type == TType && t->symbol && strcmp(t->symbol->name, "heim_any")) {
if (t->type == TType && t->symbol && strcmp(t->symbol->name, "HEIM_ANY")) {
/*
* This type is ultimately an alias of an imported type, so we don't
* know its outermost tag here.
@@ -1375,13 +1602,10 @@ generate_type_header (const Symbol *s)
return;
fprintf(headerfile, "typedef ");
define_type(0, s->gen_name, s->gen_name, s->type, TRUE,
define_type(0, s->gen_name, s->gen_name, NULL, s->type, TRUE,
preserve_type(s->name) ? TRUE : FALSE);
fprintf(headerfile, "\n");
if (template_flag)
generate_template_type_forward(s->gen_name);
emitted_definition(s);
}
@@ -1390,6 +1614,8 @@ generate_type_header_forwards(const Symbol *s)
{
declare_type(s, s->type, TRUE);
fprintf(headerfile, "\n");
if (template_flag)
generate_template_type_forward(s->gen_name);
}
void