Commit Graph

51 Commits

Author SHA1 Message Date
Volker Lendecke
f4faaeaba3 heimdal: Fix the 32-bit build on FreeBSD
Format string fixes that fail in the Samba build on a 32-bit machine

Signed-off-by: Volker Lendecke <vl@samba.org>
2022-11-16 23:46:46 -05:00
Nicolas Williams
9c9dac2b16 asn1: CVE-2022-44640 invalid free in ASN.1 codec
Heimdal's ASN.1 compiler generates code that allows specially
crafted DER encodings of CHOICEs to invoke the wrong free function
on the decoded structure upon decode error.  This is known to impact
the Heimdal KDC, leading to an invalid free() of an address partly
or wholly under the control of the attacker, in turn leading to a
potential remote code execution (RCE) vulnerability.

This error affects the DER codec for all CHOICE types used in
Heimdal, though not all cases will be exploitable.  We have not
completed a thorough analysis of all the Heimdal components
affected, thus the Kerberos client, the X.509 library, and other
parts, may be affected as well.

This bug has been in Heimdal since 2005.  It was first reported by
Douglas Bagnall, though it had been found independently by the
Heimdal maintainers via fuzzing a few weeks earlier.
2022-11-15 17:51:45 -06:00
Nicolas Williams
4748f3a19d asn1: Revert heim_object_t support 2022-01-28 17:24:57 -06:00
Nicolas Williams
9fb444983e asn1: Better handling of >63 named bits/ints
First, we enlarge Member's val field to int64_t.

Then we warn about skipping 2int, int2, and parse units glue for such
things with too-large members.

And we error out when generating the template for such things with
>UINT32_MAX members.

What about too-negative members?  That could be a thing for INTEGER /
ENUMERATED.  We'll look at that later.
2022-01-17 12:39:19 -06:00
Nicolas Williams
a31db2af0d asn1: Allow CHOICEs to be decorated too
Prior to this commit only those C structs for SET and SEQUENCE types
could be decorated.  Now those for CHOICE types also can be decorated.

We could further extend this to SET OF and SEQUENCE OF types if it
proves useful.
2022-01-15 21:09:12 -06:00
Nicolas Williams
a4e6c33780 asn1: Fix double-free in asn1parse.y 2022-01-14 22:16:50 -06:00
Nicolas Williams
1fe3d293e1 asn1: Fix warnings 2022-01-14 12:25:43 -06:00
Nicolas Williams
40d1271094 asn1: Expand decoration w/ C types
This commits allows `heim_object_t` as a type and causes the generated
code to use the `heim_retain()` and `heim_release()` functions for
copying and releasing values of such types.

Also, now one can have more than one decoration per-type.
2022-01-11 10:21:05 -06:00
Luke Howard
ef906991fd asn1: don't include decoration header for void *
If the decorated external type is a void *, don't include the decoration
header in the ASN.1 header, only in the template implementation. This allows
the copy constructor and destructor to be implementation private.
2022-01-11 09:57:11 -06:00
Nicolas Williams
df3e08485b asn1: Add support for decoration w/ external types
This adds support for asn1_compile --decorate=... variation that causes
decoration of an ASN.1 SET/SEQUENCE type with a field of a non-ASN.1
type.

This means we can now have an ASN.1 type to represent a request that can
then have a "hidden" field -- hidden in that it is neither encoded nor
decoded.  This field will be copied and freed when the decoration is of
an ASN.1 type or of a external, C type that comes with copy constructor
and destructor functions.  Decoration with a `void *` field which is
neither copied nor freed is also supported.

We may end up using this to, for example, replace the `hdb_entry_ex`
type by decorating `HDB_entry` with a C type that points to the `HDB` in
which the entry was found or to which it should be written.
2022-01-11 09:57:11 -06:00
Jeffrey Altman
fb1ded497a lib/asn1: gen_template cast integer const to uintptr_t then void ptr
When generating code cast integer constants to (void *) with an
intermediate cast to (uintptr_t).   On Windows this avoid hundreds
of warnings about a signed integer being cast to a pointer of larger
size.

No functional change.

Change-Id: Ifd2acd0cfe99a7859aad58e79ab0f0c12b28fdeb
2022-01-05 12:58:48 -06:00
Nicolas Williams
823fb82477 asn1: Add --decorate=... for internal bookkeeping
This option, `--decorate=TYPE-NAME:FIELD-TYPE:field-name[?]` allows one to add
a field to any struct generated by the ASN.1 compiler for any SET or SEQUENCE
type such that:

 - the field will     be freed by the `free_TYPE_NAME()` function
 - the field will     be copied by the `copy_TYPE_NAME()` function
 - the field will not be printed by the `print_TYPE_NAME()` function
 - the field will NOT be encoded or decoded

This is useful for internal bookkeeping.

The first use of this may well be for adding an optional field to
`Principal` where information about name attributes will be stored,
which will then allow us to have GSS name attributes for the krb5
mechanism w/o having to refactor the mechanism to use a different
structure for representing `gss_name_t` mechnames than the one currently
used (`Principal`; `krb5_principal` happens to be a typedef alias of
`Principal *`).

So w/o massive rototilling of the GSS krb5 mechanism we can have name
attributes, _and_ we'll also be able to have those in the krb5 API as
well w/o any massive rototilling there either.
2021-12-19 23:21:35 -06:00
Andrew Bartlett
7686028718 Use UTF-8 in KTH copyright notice
Samba is starting to protect against bi-di attacks and the starting point
is to require that input files be fully UTF-8.  In 2021 this is a reasonable
starting point anyway.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
2021-11-29 12:50:26 +11:00
Nicolas Williams
5f63215d0d Always perform == or != operation on cmp function result
Although not required to address bad code generation in
some versions of gcc 9 and 10, a coding style that requires
explicit comparison of the result to zero before use is
both clearer and would have avoided the generation of bad
code.

This change converts all use of cmp function usage from

```
    if (strcmp(a, b) || !strcmp(c, d)) ...
```

to

```
    if (strcmp(a, b) != 0 || strcmp(c, d)) == 0
```

for all C library cmp functions and related:

 - strcmp(), strncmp()
 - strcasecmp(), strncasecmp()
 - stricmp(), strnicmp()
 - memcmp()

Change-Id: Ic60c15e1e3a07e4faaf10648eefe3adae2543188
2021-11-24 22:30:44 -05:00
Luke Howard
89ff0eb462 asn1: ensure template generator includes ASN1CALL
Windows 32-bit builds were broken as the ASN1CALL calling convention
macro was not included in the output of gen_template.c, only in the
generated header.
2021-08-12 16:33:45 +10:00
Luke Howard
f67dd0f903 roken: move Heimdal vis.h extensions to separate header
vis.h is not built on platforms (such as macOS) that already have
that header, which resulted in prototypes for Heimdal-specific
vis.h extensions being absent.

Move those prototypes to a separate header, vis-extras.h, which
must be explicitly included in order to use the Heimdal extensions.
2021-05-17 10:05:58 +10:00
Nicolas Williams
614b3a5914 asn1: Fix OS X build
We can't use VIS_DQ on when the OS has a vis implementation but lacks
VIS_DQ.
2021-03-27 23:29:34 -05:00
Nicolas Williams
7f4e9db9f9 asn1: Make int sizing consistent and better 2021-03-10 19:15:17 -06:00
Nicolas Williams
00e0475ce2 asn1: Use name not gen_name for JSON enum printing 2021-03-07 16:32:41 -06:00
Nicolas Williams
5d7d1897de asn1: Pretty-print enum values 2021-03-07 13:36:37 -06:00
Nicolas Williams
f7a018f002 asn1: Minor compiler performance improvment 2021-03-06 15:20:42 -06:00
Nicolas Williams
3f206d56fb asn1: Add missing cases for ENUMERATED 2021-03-06 15:13:53 -06:00
Nicolas Williams
7f941b220b asn1: Basic validation of objects / object sets
- Validate that required value fields have values specified in objects

 - Validate that unique value fields have unique values within any
   object set
2021-02-28 18:15:57 -06:00
Nicolas Williams
fb2c81f3e8 asn1: Add printing of ASN.1 values
JSON-like printing of ASN.1 parsed values.

Status:

 - Not X.697 (JER) compliant, not even close.

 - String escaping isn't JSON-compliant.

 - It's not printing the names of CHOICE types.

 - It's not printing the names of open type choices (i.e., IOS object
   names)

And yet it's quite useful already.  It prints all the weird things in EK
certs for example.

Here's what it outputs for the EK cert we use in check-gen.c:

{
  "_type": "Certificate",
  "tbsCertificate": {
    "_type": "TBSCertificate",
    "_save": "30820376A00302010202146A0597BA71D7E6D3AC0EDC9EDC95A15B998DE40A300D06092A864886F70D01010B05003055310B3009060355040613024348311E301C060355040A131553544D6963726F656C656374726F6E696373204E56312630240603550403131D53544D2054504D20454B20496E7465726D656469617465204341203035301E170D3138313231343030303030305A170D3238313231343030303030305A300030820122300D06092A864886F70D01010105000382010F003082010A0282010100CC14EB27A78CEB0EA486FA2DF7835F5FA8E905B097012B5BDE50380C355B1A2A721BBC3D08DD21796CDB239FA95310651B1B56FD2CFE53C87352EBD996E33256160404CE9302A08066801E786A2F86E181F949966F492A85B58EAA4A6A8CB3697551BB236E87CC7BF8EC1347871C91E15437E8F266BF1EA5EB271FDCF374D8B47DF8BCE89E1FAD61C2A088CB4036B359CB72A294973FEDCCF0C340AFFD14B64F041165581ACA34147C1C75617047058F7ED7D603E032508094FA73E8B9153DA3BF255D2CBBC5DF301BA8F74D198BEBCE86040FC1D2927C7657414490D802F482F3EBF2DE35EE149A1A6DE8D16891FBFBA02A18AFE59F9D6F149744E5F0D559B10203010001A38201A9308201A5301F0603551D230418301680141ADB994AB58BE57A0CC9B900E7851E1A43C0866030420603551D20043B303930370604551D2000302F302D06082B060105050702011621687474703A2F2F7777772E73742E636F6D2F54504D2F7265706F7369746F72792F30590603551D110101FF044F304DA44B304931163014060567810502010C0B69643A353335343444323031173015060567810502020C0C53543333485450484148433031163014060567810502030C0B69643A303034393030303830670603551D090460305E301706056781050210310E300C0C03322E300201000202008A304306056781050212313A30380201000101FFA0030A0101A1030A0100A2030A0100A310300E1603332E310A01040A01020101FFA40F300D16053134302D320A0102010100300E0603551D0F0101FF040403020520300C0603551D130101FF0402300030100603551D250409300706056781050801304A06082B06010505070101043E303C303A06082B06010505073002862E687474703A2F2F7365637572652E676C6F62616C7369676E2E636F6D2F73746D74706D656B696E7430352E637274",
    "version": "2",
    "serialNumber": "6A0597BA71D7E6D3AC0EDC9EDC95A15B998DE40A",
    "signature": {
      "_type": "AlgorithmIdentifier",
      "algorithm": "1.2.840.113549.1.1.11",
      "parameters": "0500"
    },
    "issuer": {
      "_choice": "rdnSequence",
      "value": [
        [
          {
            "_type": "AttributeTypeAndValue",
            "type": "2.5.4.6",
            "value": {
              "_choice": "printableString",
              "value": "CH"
            }
          }
        ],
        [
          {
            "_type": "AttributeTypeAndValue",
            "type": "2.5.4.10",
            "value": {
              "_choice": "printableString",
              "value": "STMicroelectronics NV"
            }
          }
        ],
        [
          {
            "_type": "AttributeTypeAndValue",
            "type": "2.5.4.3",
            "value": {
              "_choice": "printableString",
              "value": "STM TPM EK Intermediate CA 05"
            }
          }
        ]
      ]
    },
    "validity": {
      "_type": "Validity",
      "notBefore": {
        "_choice": "utcTime",
        "value": "2018-12-14T00:00:00Z"
      },
      "notAfter": {
        "_choice": "utcTime",
        "value": "2028-12-14T00:00:00Z"
      }
    },
    "subject": {
      "_choice": "rdnSequence",
      "value": []
    },
    "subjectPublicKeyInfo": {
      "_type": "SubjectPublicKeyInfo",
      "algorithm": {
        "_type": "AlgorithmIdentifier",
        "algorithm": "1.2.840.113549.1.1.1",
        "parameters": "0500"
      },
      "subjectPublicKey": "2160:3082010A0282010100CC14EB27A78CEB0EA486FA2DF7835F5FA8E905B097012B5BDE50380C355B1A2A721BBC3D08DD21796CDB239FA95310651B1B56FD2CFE53C87352EBD996E33256160404CE9302A08066801E786A2F86E181F949966F492A85B58EAA4A6A8CB3697551BB236E87CC7BF8EC1347871C91E15437E8F266BF1EA5EB271FDCF374D8B47DF8BCE89E1FAD61C2A088CB4036B359CB72A294973FEDCCF0C340AFFD14B64F041165581ACA34147C1C75617047058F7ED7D603E032508094FA73E8B9153DA3BF255D2CBBC5DF301BA8F74D198BEBCE86040FC1D2927C7657414490D802F482F3EBF2DE35EE149A1A6DE8D16891FBFBA02A18AFE59F9D6F149744E5F0D559B10203010001"
    },
    "issuerUniqueID": null,
    "subjectUniqueID": null,
    "extensions": [
      {
        "_type": "Extension",
        "extnID": "2.5.29.35",
        "critical": false,
        "extnValue": "301680141ADB994AB58BE57A0CC9B900E7851E1A43C08660",
        "_extnValue": {
          "_type": "AuthorityKeyIdentifier",
          "keyIdentifier": "1ADB994AB58BE57A0CC9B900E7851E1A43C08660",
          "authorityCertIssuer": null,
          "authorityCertSerialNumber": null
        }
      },
      {
        "_type": "Extension",
        "extnID": "2.5.29.32",
        "critical": false,
        "extnValue": "303930370604551D2000302F302D06082B060105050702011621687474703A2F2F7777772E73742E636F6D2F54504D2F7265706F7369746F72792F",
        "_extnValue": [
          {
            "_type": "PolicyInformation",
            "policyIdentifier": "2.5.29.32.0",
            "policyQualifiers": [
              {
                "_type": "PolicyQualifierInfo",
                "policyQualifierId": "1.3.6.1.5.5.7.2.1",
                "qualifier": "1621687474703A2F2F7777772E73742E636F6D2F54504D2F7265706F7369746F72792F"
              }
            ]
          }
        ]
      },
      {
        "_type": "Extension",
        "extnID": "2.5.29.17",
        "critical": true,
        "extnValue": "304DA44B304931163014060567810502010C0B69643A353335343444323031173015060567810502020C0C53543333485450484148433031163014060567810502030C0B69643A3030343930303038",
        "_extnValue": [
          {
            "_choice": "directoryName",
            "value": {
              "_choice": "rdnSequence",
              "value": [
                [
                  {
                    "_type": "AttributeTypeAndValue",
                    "type": "2.23.133.2.1",
                    "value": {
                      "_choice": "utf8String",
                      "value": "id:53544D20"
                    }
                  }
                ],
                [
                  {
                    "_type": "AttributeTypeAndValue",
                    "type": "2.23.133.2.2",
                    "value": {
                      "_choice": "utf8String",
                      "value": "ST33HTPHAHC0"
                    }
                  }
                ],
                [
                  {
                    "_type": "AttributeTypeAndValue",
                    "type": "2.23.133.2.3",
                    "value": {
                      "_choice": "utf8String",
                      "value": "id:00490008"
                    }
                  }
                ]
              ]
            }
          }
        ]
      },
      {
        "_type": "Extension",
        "extnID": "2.5.29.9",
        "critical": false,
        "extnValue": "305E301706056781050210310E300C0C03322E300201000202008A304306056781050212313A30380201000101FFA0030A0101A1030A0100A2030A0100A310300E1603332E310A01040A01020101FFA40F300D16053134302D320A0102010100",
        "_extnValue": [
          {
            "_type": "AttributeSet",
            "type": "2.23.133.2.16",
            "values": [
              "300C0C03322E300201000202008A"
            ],
            "_values": [
              {
                "_type": "TPMSpecification",
                "family": "2.0",
                "level": "0",
                "revision": "138"
              }
            ]
          },
          {
            "_type": "AttributeSet",
            "type": "2.23.133.2.18",
            "values": [
              "30380201000101FFA0030A0101A1030A0100A2030A0100A310300E1603332E310A01040A01020101FFA40F300D16053134302D320A0102010100"
            ],
            "_values": [
              {
                "_type": "TPMSecurityAssertions",
                "version": "0",
                "fieldUpgradable": true,
                "ekGenerationType": "655617",
                "ekGenerationLocation": "655616",
                "ekCertificateGenerationLocation": "655616",
                "ccInfo": {
                  "_type": "CommonCriteriaMeasures",
                  "version": "3.1",
                  "assurancelevel": "4",
                  "evaluationStatus": "2",
                  "plus": true,
                  "strengthOfFunction": null,
                  "profileOid": null,
                  "profileUri": null,
                  "targetOid": null,
                  "targetUri": null
                },
                "fipsLevel": {
                  "_type": "FIPSLevel",
                  "version": "140-2",
                  "level": "2",
                  "plus": false
                },
                "iso9000Certified": false,
                "iso9000Uri": null
              }
            ]
          }
        ]
      },
      {
        "_type": "Extension",
        "extnID": "2.5.29.15",
        "critical": true,
        "extnValue": "03020520",
        "_extnValue": [
          "keyEncipherment"
        ]
      },
      {
        "_type": "Extension",
        "extnID": "2.5.29.19",
        "critical": true,
        "extnValue": "3000",
        "_extnValue": {
          "_type": "BasicConstraints",
          "cA": false,
          "pathLenConstraint": null
        }
      },
      {
        "_type": "Extension",
        "extnID": "2.5.29.37",
        "critical": false,
        "extnValue": "300706056781050801",
        "_extnValue": [
          "2.23.133.8.1"
        ]
      },
      {
        "_type": "Extension",
        "extnID": "1.3.6.1.5.5.7.1.1",
        "critical": false,
        "extnValue": "303C303A06082B06010505073002862E687474703A2F2F7365637572652E676C6F62616C7369676E2E636F6D2F73746D74706D656B696E7430352E637274",
        "_extnValue": [
          {
            "_type": "AccessDescription",
            "accessMethod": "1.3.6.1.5.5.7.48.2",
            "accessLocation": {
              "_choice": "uniformResourceIdentifier",
              "value": "http://secure.globalsign.com/stmtpmekint05.crt"
            }
          }
        ]
      }
    ]
  },
  "signatureAlgorithm": {
    "_type": "AlgorithmIdentifier",
    "algorithm": "1.2.840.113549.1.1.11",
    "parameters": "0500"
  },
  "signatureValue": "2048:3D4C381E5B4F1BCBE09C63D52F1F04570CAEA142FD9CD942043B11F8E3BDCF50007AE16CF8869013041E92CDD3280BA4B51FBBD40582ED750219E261A695095674855AACEB520ADAFF9E7E908480A39CDCF900462D9171960FFE55D3AC49E8C981341BBD2EFBCC252A4C18A4F3B7C84CCE42CE70A208C84D2630A7ABFBE72D6271E75B9FF1C971D20EB3DBD763F1E04D834EAA692D2E4001BBF4730A3E3FDA9711AE386524D91C63BE0E516D00D5C6141FCCF6C539F3518E180049865BE16B69CAE1F8CB7FDC474B38F7EE56CBE7D8A89D9BA99B65D5265AEF32AA62426B10E6D75BB8677EC44F755BBC2806FD2B4E04BDF5D44259DBEAA42B6F563DF7AA7506"
}
2021-02-28 18:13:08 -06:00
Nicolas Williams
db7763ca7b asn1: X.681/682/683 magic handling of open types
Status:

 - And it works!

 - We have an extensive test based on decoding a rich EK certficate.

   This test exercises all of:

    - decoding
    - encoding with and without decoded open types
    - copying of decoded values with decoded open types
    - freeing of decoded values with decoded open types

   Valgrind finds no memory errors.

 - Added a manual page for the compiler.

 - rfc2459.asn1 now has all three primary PKIX types that we care about
   defined as in RFC5912, with IOS constraints and parameterization:

    - `Extension`       (embeds open type in an `OCTET STRING`)
    - `OtherName`       (embeds open type in an        `ANY`-like type)
    - `SingleAttribute` (embeds open type in an        `ANY`-like type)
    - `AttributeSet`    (embeds open type in a  `SET OF ANY`-like type)

   All of these use OIDs as the open type type ID field, but integer
   open type type ID fields are also supported (and needed, for
   Kerberos).

   That will cover every typed hole pattern in all our ASN.1 modules.

   With this we'll be able to automatically and recursively decode
   through all subject DN attributes even when the subject DN is a
   directoryName SAN, and subjectDirectoryAttributes, and all
   extensions, and all SANs, and all authorization-data elements, and
   PA-data, and...

   We're not really using `SingleAttribute` and `AttributeSet` yet
   because various changes are needed in `lib/hx509` for that.

 - `asn1_compile` builds and recognizes the subset of X.681/682/683 that
   we need for, and now use in, rfc2459.asn1.  It builds the necessary
   AST, generates the correct C types, and generates templating for
   object sets and open types!

 - See READMEs for details.

 - Codegen backend not tested; I won't make it implement automatic open
   type handling, but it should at least not crash by substituting
   `heim_any` for open types not embedded in `OCTET STRING`.

 - We're _really_ starting to have problems with the ITU-T ASN.1
   grammar and our version of it...

   Type names have to start with upper-case, value names with
   lower-case, but it's not enough to disambiguate.

   The fact the we've allowed value and type names to violate their
   respective start-with case rules is causing us trouble now that we're
   adding grammar from X.681/682/683, and we're going to have to undo
   that.

   In preparation for that I'm capitalizing the `heim_any` and
   `heim_any_set` types, and doing some additional cleanup, which
   requires changes to other parts of Heimdal (all in this same commit
   for now).

   Problems we have because of this:

    - We cannot IMPORT values into modules because we have no idea if a
      symbol being imported refers to a value or a type because the only
      clue we would have is the symbol's name, so we assume IMPORTed
      symbols are for types.

      This means we can't import OIDs, for example, which is super
      annoying.

      One thing we might be able to do here is mark imported symbols as
      being of an undetermined-but-not-undefined type, then coerce the
      symbol's type the first time it's used in a context where its type
      is inferred as type, value, object, object set, or class.  (Though
      since we don't generate C symbols for objects or classes, we won't
      be able to import them, especially since we need to know them at
      compile time and cannot defer their handling to link- or
      run-time.)

    - The `NULL` type name, and the `NULL` value name now cause two
      reduce/reduce conflicts via the `FieldSetting` production.

    - Various shift/reduce conflicts involving `NULL` values in
      non-top-level contexts (in constraints, for example).

 - Currently I have a bug where to disambiguate the grammar I have a
   CLASS_IDENTIFIER token that is all caps, while TYPE_IDENTIFIER must
   start with a capital but not be all caps, but this breaks Kerberos
   since all its types are all capitalized -- oof!

   To fix this I made it so class names have to be all caps and
   start with an underscore (ick).

TBD:

 - Check all the XXX comments and address them
 - Apply this treatment to Kerberos!  Automatic handling of authz-data
   sounds useful :)
 - Apply this treatment to PKCS#10 (CSRs) and other ASN.1 modules too.
 - Replace various bits of code in `lib/hx509/` with uses of this
   feature.
 - Add JER.
 - Enhance `hxtool` and `asn1_print`.

Getting there!
2021-02-28 18:13:08 -06:00
Nicolas Williams
89f97e8287 asn1: Improve XXX comment 2021-02-25 17:33:18 -06:00
Nicolas Williams
783b632f1f asn1: Teach template backend to DEFAULT 2021-02-01 22:30:33 -06:00
Nicolas Williams
3da24c19ad asn1: Template backend partial support for SETs
This adds enough support for SET { ... } types to the template backend
to allow the X.690 sample test to be run with the template backend.

Limitations:

 - For DER encoding, the SET { ... } members must be manually sorted by
   the module author.

 - Decoding of out-of-order encodings (BER) is not supported at this
   time.

These shortcomings will be addressed later.

Note that because we don't parse IMPORTed modules at this time, we can't
sort SET { ... } members at compile time if any of them out an
outer-most tag that the compiler cannot see without learning to parse
IMPORTed modules.
2021-02-01 22:30:33 -06:00
Nicolas Williams
8fde460772 asn1: More IMPLICIT tag fixes (both compilers)
The template compiler was applying IMPLICIT tags to CHOICE types.  This
is very wrong, as the tag of a CHOICE's taken choice cannot be replaced
without making it impossible to figure out what the choice was.  An
example of this is GeneralName's directoryName, which is an IMPLICIT-
tagged CHOICE.

Separately, the non-template compiler was requiring inlining of
IMPLICIT-tagged CHOICEs, which also happens in GeneralName's
directoryName case:

```
    205 Name ::= CHOICE {
    206         rdnSequence  RDNSequence
    207 }
    ...
    287 GeneralName ::= CHOICE {
    288         otherName                       [0]     IMPLICIT -- OtherName --
    SEQUENCE {
    289                 type-id    OBJECT IDENTIFIER,
    290                 value      [0] EXPLICIT heim_any
    291         },
    292         rfc822Name                      [1]     IMPLICIT IA5String,
    293         dNSName                         [2]     IMPLICIT IA5String,
    294 --      x400Address                     [3]     IMPLICIT ORAddress,--
--->295         directoryName                   [4]     IMPLICIT -- Name -- CHOICE
    {
    296                 rdnSequence  RDNSequence
    297         },
    298 --      ediPartyName                    [5]     IMPLICIT EDIPartyName, --
    299         uniformResourceIdentifier       [6]     IMPLICIT IA5String,
    300         iPAddress                       [7]     IMPLICIT OCTET STRING,
    301         registeredID                    [8]     IMPLICIT OBJECT IDENTIFIER
    302 }
```

Anyways, that's fixed now, though changing that will require making
corresponding changes to `lib/hx509/`.

We're getting closer to parity between the two compilers.  The template
compiler is still missing support for `SET { ... }` types.  Speaking of
`SET { ... }`, the regular compiler generates code that uses `qsort()`
to sort the encoded values values of the members of such a set, but this
seems silly because the order of members is knowable at compile time, as
for DER and CER the order by the tags of the members, from lowest to
highest (see X.690, section 9.3 and X.680, section 8.6).  As it happens
using `qsort()` on the encodings of the members works, but it would be
be better to sort in `lib/asn1/asn1parse.y` and then not have to bother
anywhere else.  Sorting SETs at definition time will help keep the
tamplate compiler simple.  Not that we _need_ `SET { ... }` for anything
in-tree other than the X.690 sample...

While we're at it, let's note that the core of PKIX from the RFC
2459/3280/5280/5912 consists of *two* ASN.1 modules, one with
default-EXPLICIT tags, and one with default-IMPLICIT tags, and
Heimdal has these merged as a default-EXPLICIT tags module in
`lib/asn1/rfc2459.asn1`, with `IMPLICIT` added in by hand in all the
tags in the default-IMPLICIT tagged module.  This fixes one recently
added type from PKIX that didn't have `IMPLICIT` added in manually!
2021-01-24 20:24:01 -06:00
Nicolas Williams
0729692cc8 asn1: Templates work for IMPLICIT; add build opt
Finally.  We're almost at parity for the template compiler.

Now we have a build option to use templating:

    `./configure --enable-asn1-templating`

Tests fail if you build `rfc2459.asn1` with `--template`.

TBD: Figure out what differences remain between the two compilers, and
     fix the templating compiler accordingly, adding tests along the
     way.

Making IMPLICIT tags work in the templating compiler turned out to be a
simple fix: don't attempt to do anything clever about IMPLICIT tags in
the template generator in the compiler other than denoting them --
instead leave all the smarts about IMPLICIT tags to the interpreter.
This might be a very slight pessimization, but also a great
simplification.

The result is very elegant: when the interpreter finds an IMPLICIT
tag it then recurses to find the template for the body of the type
so-tagged, and evaluates that.  Much more elegant than the code
generated by the non-template compiler, not least for not needing
any additional temporary memory allocation.

With this we finally have parity in basic testing of the template
compiler.  Indeed, for IMPLICIT tags the template compiler and
interpreter might even be better because they support IMPLICIT tags
with BER lengths, whereas the non-template compiler doesn't (mostly
because `der_replace_tag()` needs to be changed to support it.

And, of course, the template compiler is simply superior in that it
produces smaller code and is *much* easier to work with because the
functions to interpret templates are small and simple.  Which means we
can add more functions to deal with other encoding rules fairly
trivially.  It should be possible to add all of these with very little
work, almost all of it localized to `lib/asn1/template.c`:

 - PER  Packed Encoding Rules [X.691]
 - XER  XML Encoding Rules    [X.693]
 - OER  Octet Encoding Rules  [X.696] (intended to replace PER)
 - JER  JSON Encoding Rules   [X.697] (doubles as visual representation)
 - GSER Generic String E.R.s  [RFC3641] (a visual representation)

 - XDR  External Data Repr.   [STD67][RFC4506]

       (XDR is *not* an ASN.1 encoding rules specification, but it's a
        *lot* like PER/OER but with 4-octet alignment, and is specified
        for the syntax equivalent (XDR) of only a subset of ASN.1 syntax
        and semantics.)

All we'd have to do is add variants of `_asn1_{length,encode,decode}()`
for each set of rules, then generate per-type stub functions that call
them (as we already do for DER).

We could then have an encoding rule transliteration program that takes a
`TypeName` and some representation of a value encoded by some encoding
rules, and outputs the same thing encoded by a different set of rules.
This would double as a pretty-printer and parser if we do add support
for JER and/or GSER.  It would find the template for the given type
using `dlsym()` against some shared object (possibly `libasn1` itself).

Whereas generating source code for C (or whatever language) for
additional ERs requires much more work.  Plus, templates are much
smaller, and the interpreter is tiny, which yields much smaller text and
much smaller CPU icache/dcache footprint, which yields better
performance in many cases.

As well, the template system should be much easier to port to other
languages.  Though in the cases of, e.g., Rust, it would require use of
`unsafe` in the interpreter, so in fact the inverse might be true: that
it's easier to generate safe Rust code than to implement a template
interpreter in Rust.  Similarly for Haskell, OCAML, etc.  But wherever
the template interpreter is easy to implement, it's a huge win.

Note that implementing OER and PER using the templates as they are
currently would be a bit of a challenge, as the interpreter would have
to first do a pass of each SEQUENCE/SET to determine the size and
layout of the OER/PER sequence/set preamble by counting the number of
OPTIONAL/DEFAULT members, BOOLEAN members, and extensibility markers
with extensions present.  We could always generate more entries to
encode precomputed preamble metadata.  We would also need to add a
template entry type for extensibility markers, which currently we do
not.
2021-01-23 17:48:12 -06:00
Nicolas Williams
89389bc7a0 asn1: Fix long-standing IMPLICIT tagging brokenness
This commit _mostly_ fixes the Heimdal ASN.1 compiler to properly
support IMPLICIT tagging in most if not all the many cases where it
didn't already, as you could see in lib/asn1/canthandle.asn1 prior to
this commit.

This fix is a bit of a hack in that a proper fix would change the
function prototypes of the encode/decode/length functions generated by
the compiler to take an optional IMPLICIT tag to tag with instead of the
type they code.  That fix would not be localized to lib/asn1/ however,
and would change the API and ABI of generated code (which is mostly not
an ABI for Heimdal, but still, some external projects would have to make
changes).

Instead, for IMPLICIT tags we currently depend on the IMPLICIT tag and
the sub-type's tag having the same size -- this can be fixed with extra
allocation on the encoder side as we do on the decoder side, but we
might leave it for later.

The issue we're fixing manifested as:

  -- The [CONTEXT 0] tag in Bar below was turned into an EXPLICIT tag
  -- instead of an IMPLICIT one, netting the DER encoding for the `foo`
  -- member as:
  --    [CONTEXT 0] [UNIVERSAL Seq] [UNIVERSAL Int] <encoding of i>
  -- instead of the correct:
  --    [CONTEXT 0] [UNIVERSAL Int] <encoding of i>
  Foo ::= SEQUENCE { i INTEGER }
  Bar ::= SEQUENCE { foo [0] IMPLICIT Foo }

or

  Foo ::= INTEGER
  Bar ::= SEQUENCE { foo [0] IMPLICIT Foo } -- tag context 0 marked
                                            -- constructed!

I've reviewed this in part by reviewing the output of the compiler
before and after this change using this procedure:

 - Run an earlier version of the ASN.1 compiler output for all
   modules in lib/asn1/.  Save these in a different location.

 - Run this (or later) version of the ASN.1 compiler output for
   the same modules, adding --original-order for modules that
   have been manually sorted already (e.g., rfc2459.asn1).

 - Run clang-format on the saved and newest generated C source
   and header files.

 - Diff the generated output.  Substantial differences will
   relate to handling of IMPLICIT tagging.  These are
   particularly evident in the tcg.asn1 module, which uses a lot
   of those.

Later commits add test data (certificates with extensions that use
IMPLICIT tagging) taken from external specifications as well, which
exercise this fix.

Non-urgent brokenness yet to be fixed:

 - When the IMPLICIT tag and the tag of the underlying type require
   differing numbers of bytes to encode, the encoding and decoding will
   fail.  The prototypes of generated length_*() functions make it
   impossible to do much better.

 - SET OF <primitive> still crashes the compiler (not a new bug).

Futures:

 - Unwind hackery in cms.asn1 that worked around our lack of proper
   IMPLICIT tagging support.

Here are some of the generated code deltas one expects to see around
this commit:

$ git checkout $earlier_version
$ ./autogen.sh
$ mkdir build
$ cd build
$ ../configure ...
$ make -j4
$ make check
$ cd lib/asn1
$ for i in *.c; do
      [[ $i = asn1parse.? || $i = lex.? || $i = *.h ]] && continue
      clang-format -i $i $i
      cmp /tmp/save/$i $i && echo NO DIFFS: $i && continue; echo DIFF: $i
  done
NO DIFFS: asn1_cms_asn1.c
NO DIFFS: asn1_digest_asn1.c
NO DIFFS: asn1_err.c
NO DIFFS: asn1_krb5_asn1.c
/tmp/save/asn1_kx509_asn1.c asn1_kx509_asn1.c differ: byte 6433, line 264
DIFF: asn1_kx509_asn1.c
NO DIFFS: asn1_ocsp_asn1.c
NO DIFFS: asn1_pkcs10_asn1.c
/tmp/save/asn1_pkcs12_asn1.c asn1_pkcs12_asn1.c differ: byte 12934, line 455
DIFF: asn1_pkcs12_asn1.c
NO DIFFS: asn1_pkcs8_asn1.c
NO DIFFS: asn1_pkcs9_asn1.c
NO DIFFS: asn1_pkinit_asn1.c
/tmp/save/asn1_rfc2459_asn1.c asn1_rfc2459_asn1.c differ: byte 20193, line 532
DIFF: asn1_rfc2459_asn1.c
NO DIFFS: asn1_rfc4043_asn1.c
/tmp/save/asn1_rfc4108_asn1.c asn1_rfc4108_asn1.c differ: byte 595, line 26
DIFF: asn1_rfc4108_asn1.c
/tmp/save/asn1_tcg_asn1.c asn1_tcg_asn1.c differ: byte 31835, line 1229
DIFF: asn1_tcg_asn1.c
/tmp/save/asn1_test_asn1.c asn1_test_asn1.c differ: byte 384, line 21
DIFF: asn1_test_asn1.c
/tmp/save/test_template_asn1-template.c test_template_asn1-template.c differ: byte 650, line 20
DIFF: test_template_asn1-template.c
$
$ cd ../..
$ git checkout $newer_version
$ make -j4 && make check
$ cd lib/asn1
$ for i in *.[ch]; do
    [[ $i = asn1parse.? || $i = lex.? || $i = *.h ]] && continue
    clang-format -i $i $i
    cmp /tmp/save/$i $i && echo NO DIFFS: $i && continue
    diff -ubw /tmp/save/$i $i
  done | $PAGER

and one should see deltas such as the following:

 - a small enhancement to handling of OPTIONAL members:

     (data)->macData = calloc(1, sizeof(*(data)->macData));
     if ((data)->macData == NULL)
       goto fail;
     e = decode_PKCS12_MacData(p, len, (data)->macData, &l);
-    if (e) {
+    if (e == ASN1_MISSING_FIELD) {
       free((data)->macData);
       (data)->macData = NULL;
+    } else if (e) {
+      goto fail;
     } else {
       p += l;
       len -= l;
       ret += l;

 - more complete handling of DEFAULTed members:

     e = decode_FWReceiptVersion(p, len, &(data)->version, &l);
-    if (e)
+    if (e == ASN1_MISSING_FIELD) {
+      (data)->version = 1;
+    } else if (e) {
       goto fail;
-    p += l;
-    len -= l;
-    ret += l;
+    } else {
+      p += l;
+      len -= l;
+      ret += l;
+    }
     {

 - replacement of tags with implicit tags (encode side):

   /* targetUri */
   if ((data)->targetUri) {
     size_t Top_tag_oldret HEIMDAL_UNUSED_ATTRIBUTE = ret;
     ret = 0;
     e = encode_URIReference(p, len, (data)->targetUri, &l);
     if (e)
       return e;
     p -= l;
     len -= l;
     ret += l;

-    e = der_put_length_and_tag(p, len, ret, ASN1_C_CONTEXT, PRIM, 4, &l);
+    e = der_replace_tag(p, len, ASN1_C_CONTEXT, CONS, 4);
     if (e)
       return e;
     p -= l;
     len -= l;
     ret += l;

     ret += Top_tag_oldret;
   }

 - replacement of tags with implicit tags (decode side):

         strengthOfFunction_oldlen = len;
         if (strengthOfFunction_datalen > len) {
           e = ASN1_OVERRUN;
           goto fail;
         }
         len = strengthOfFunction_datalen;
-        e = decode_StrengthOfFunction(p, len, (data)->strengthOfFunction, &l);
-        if (e)
-          goto fail;
-        p += l;
-        len -= l;
-        ret += l;
+        {
+          unsigned char *pcopy;
+          pcopy = calloc(1, len);
+          if (pcopy == 0) {
+            e = ENOMEM;
+            goto fail;
+          }
+          memcpy(pcopy, p, len);
+          e = der_replace_tag(pcopy, len, ASN1_C_UNIV, PRIM, 0);
+          if (e)
+            goto fail;
+          e = decode_StrengthOfFunction(p, len, (data)->strengthOfFunction, &l);
+          if (e)
+            goto fail;
+          p += l;
+          len -= l;
+          ret += l;
+          free(pcopy);
+        }
         len = strengthOfFunction_oldlen - strengthOfFunction_datalen;
       }
     }
     {
       size_t profileOid_datalen, profileOid_oldlen;

 - correct determination of implicit tag constructed vs no for IMPLICT-
   tagged named primitive types:

     {
       size_t profileUri_datalen, profileUri_oldlen;
       Der_type profileUri_type;
       e = der_match_tag_and_length(p, len, ASN1_C_CONTEXT, &profileUri_type, 2,
                                    &profileUri_datalen, &l);
-      if (e == 0 && profileUri_type != PRIM) {
+      if (e == 0 && profileUri_type != CONS) {
         e = ASN1_BAD_ID;
       }
       if (e) {
         (data)->profileUri = NULL;
       } else {
         (data)->profileUri = calloc(1, sizeof(*(data)->profileUri));
         if ((data)->profileUri == NULL) {
           e = ENOMEM;
           goto fail;
         }

 - correct determination of length of IMPLICT-tagged OIDs:

   if ((data)->profileOid) {
     size_t Top_tag_oldret = ret;
     ret = 0;
     ret += der_length_oid((data)->profileOid);
+    ret += 1 + der_length_len(ret);
     ret += Top_tag_oldret;
   }

These deltas should be examined with the corresponding ASN.1 module at
hand, cross-referencing the source code to the ASN.1 type definitions
and manually applying X.690 rules to double-check the choices of
primitive vs. constructed tag, and the choices of when to replace tags
and when not.
2021-01-13 20:17:58 -06:00
Nicolas Williams
83d4c6ddb5 asn1: Circular types and Topo. sort declarations
Many external ASN.1 modules that we have imported over time define types
like this:

  Foo ::= SEQUENCE { bar Bar }
  Bar ::= SEQUENCE { aMember INTEGER }

and before this change one had to re-order the definitions so that the
one for `Bar` came first.  No more.

We can now have out of order definitions in ASN.1 modules and the
compiler will topologically sort output C type declarations so that one
no longer has to manually sort types in ASN.1 modules when importing
them.

Besides that, it is now possible to create circular data types using
OPTIONAL since we generate such fields as pointers (which can then be
pointers to incomplete struct declarations):

  Circular ::= SEQUENCE {
          name UTF8String,
          next Circular OPTIONAL
  }

Circular types aren't necessarily useful, but they have been used in the
past.  E.g., the rpc.mountd protocol uses a circular type as a linked
list -- it should just have used an array, of course, as that's
semantically equivalent but more space efficient in its encoding, but
the point is that such types exist out there.
2021-01-13 20:17:58 -06:00
Nicolas Williams
45451814b3 asn1: Add breadcrumbs for future work 2020-12-16 15:12:00 -06:00
Nicolas Williams
473e5be741 asn1: Leave comment about great futures 2020-09-07 22:04:59 -05:00
Luke Howard
d7138cfbe7 base: make heimqueue.h a shared header
Share heimqueue.h between base and asn1, to avoid duplication of code.
2020-02-04 17:28:35 +11:00
Nicolas Williams
6a0f45c4d7 Use __attribute__ ((__name__)) form
Protect against macros named noreturn and so on.
2017-03-13 18:39:41 -04:00
Viktor Dukhovni
3d590d651f Reapply incorectly reverted gen_template bugfix
Without this, template memory allocation is incorrect for nested
sequences, which, as luck would have it, breaks tests on NetBSD
(whose malloc seems to give tighter allocations).

This partly undoes:

    commit 060474df16
    Author: Love Hornquist Astrand <lha@h5l.org>
    Date:   Mon Jun 3 21:45:51 2013 -0700

	quel 64bit warnings, fixup implicit encoding for template,
	fix spelling

Restoring changes from:

    commit 5e081aa4a6
    Author: Viktor Dukhovni <viktor@dukhovni.org>
    Date:   Sun May 27 08:07:28 2012 +0000

	Fix ASN.1 template compiler bug and add test cases more
	likely to trip on similar (structure size/type) errors

For example, without the bugfix, the sizeof(...) argument in multiple
generated nested structure templates is wrong, as seen in the bad vs.
good diff:

    --- test_template_asn1-template.c	2016-11-09 08:23:21.000000000 +0000
    +++ test_template_asn1-template.c	2016-11-09 08:23:40.000000000 +0000
    @@ -593,3 +593,3 @@
     const struct asn1_template asn1_TESTImplicit_tag_ti2_26[] = {
    -/* 0 */ { 0, sizeof(struct TESTImplicit), ((void *)1) },
    +/* 0 */ { 0, sizeof(struct TESTImplicit_ti2), ((void *)1) },
     /* 1 */ { A1_TAG_T(ASN1_C_CONTEXT,CONS,127), offsetof(struct TESTImplicit_ti2, foo), asn1_TESTLargeTag_tag_foo_4 }
    @@ -1618,3 +1618,3 @@
     const struct asn1_template asn1_TESTSeqOf2_tag_strings_68[] = {
    -/* 0 */ { 0, sizeof(struct TESTSeqOf2), ((void *)1) },
    +/* 0 */ { 0, sizeof(struct TESTSeqOf2_strings), ((void *)1) },
     /* 1 */ { A1_OP_SEQOF, 0, asn1_TESTSeqOfSeq2_val_tag_string_60 }
    @@ -1679,3 +1679,3 @@
     const struct asn1_template asn1_TESTSeqOf3_tag_strings_71[] = {
    -/* 0 */ { 0, sizeof(struct TESTSeqOf3), ((void *)1) },
    +/* 0 */ { 0, sizeof(struct TESTSeqOf3_strings), ((void *)1) },
     /* 1 */ { A1_OP_SEQOF, 0, asn1_TESTSeqOfSeq2_val_tag_string_60 }
    @@ -1760,3 +1760,3 @@
     const struct asn1_template asn1_TESTSeqOf4_tag_b1_75[] = {
    -/* 0 */ { 0, sizeof(struct TESTSeqOf4), ((void *)1) },
    +/* 0 */ { 0, sizeof(struct TESTSeqOf4_b1), ((void *)1) },
     /* 1 */ { A1_OP_SEQOF, 0, asn1_TESTSeqOf4_seofTstruct_10 }
    @@ -1765,3 +1765,3 @@
     const struct asn1_template asn1_TESTSeqOf4_tag_b1_74[] = {
    -/* 0 */ { 0, sizeof(struct TESTSeqOf4), ((void *)1) },
    +/* 0 */ { 0, sizeof(struct TESTSeqOf4_b1), ((void *)1) },
     /* 1 */ { A1_TAG_T(ASN1_C_UNIV,CONS,UT_Sequence), 0, asn1_TESTSeqOf4_tag_b1_75 }
    @@ -1801,3 +1801,3 @@
     const struct asn1_template asn1_TESTSeqOf4_tag_b2_79[] = {
    -/* 0 */ { 0, sizeof(struct TESTSeqOf4), ((void *)1) },
    +/* 0 */ { 0, sizeof(struct TESTSeqOf4_b2), ((void *)1) },
     /* 1 */ { A1_OP_SEQOF, 0, asn1_TESTSeqOf4_seofTstruct_11 }
    @@ -1842,3 +1842,3 @@
     const struct asn1_template asn1_TESTSeqOf4_tag_b3_84[] = {
    -/* 0 */ { 0, sizeof(struct TESTSeqOf4), ((void *)1) },
    +/* 0 */ { 0, sizeof(struct TESTSeqOf4_b3), ((void *)1) },
     /* 1 */ { A1_OP_SEQOF, 0, asn1_TESTSeqOf4_seofTstruct_12 }
2016-11-09 03:33:34 -05:00
Love Hornquist Astrand
060474df16 quel 64bit warnings, fixup implicit encoding for template, fix spelling 2013-06-03 21:46:20 -07:00
Stefan Metzmacher
d28613ce11 heimdal:lib/asn1: try to fix the build on IRIX
cc-1028 cc: ERROR File = ../source4/heimdal/lib/asn1/gen_template.c, Line = 548
  The expression used must have a constant value.

  struct templatehead template = { 0L, &(template). tqh_first };
                                         ^
If this really fixes the IRIX build, we'll propose this for heimdal upstream.

metze

Signed-off-by: Love Hornquist Astrand <lha@h5l.org>
2012-06-11 11:05:37 +02:00
Roland C. Dowdeswell
abdde6a608 Additional changes to make -Wshadow build on Ubuntu 10.04.
Looks like they defined basename() in string.h and ntohs/htonl are
implemented in terms of __bswap16() which is a macro with tmp
variables and so one cannot embed one call to ntohs/htons in another.
Not good but we workaround this limitation in glibc.
2012-06-07 16:59:01 +01:00
Viktor Dukhovni
5e081aa4a6 Fix ASN.1 template compiler bug and add test cases more likely to trip on similar (structure size/type) errors
Signed-off-by: Roland C. Dowdeswell <elric@imrryr.org>
2012-06-05 22:05:35 +01:00
Roland C. Dowdeswell
e8779d5d4a Add -Wshadow and deal with the warnings. 2012-02-21 11:17:55 +00:00
Nicolas Williams
a222521e68 64-bit build fixes for ASN.1 compiler 64-bit integer support 2011-12-13 13:03:57 -06:00
Nicolas Williams
19d378f44d Add 64-bit integer support to ASN.1 compiler
ASN.1 INTEGERs will now compile to C int64_t or uint64_t, depending
    on whether the constraint ranges include numbers that cannot be
    represented in 32-bit ints and whether they include negative
    numbers.

    Template backend support included.  check-template is now built with
    --template, so we know we're testing it.

    Tests included.
2011-12-12 20:01:20 -06:00
Love Hornquist Astrand
0879b9831a remove trailing whitespace 2011-05-21 11:57:31 -07:00
Jeffrey Altman
6850d6a65f avoid uninit variable and unreachable code warnings
most of these warnings are not problems because of ample
use of abort() calls.  However, the large number of warnings
makes it difficult to identify real problems.  Initialize
the variables to shut up the compilers.

Change-Id: I8477c11b17c7b6a7d9074c721fdd2d7303b186a8
2011-05-17 12:02:16 -04:00
Jeffrey Altman
36dcd37cc7 use %p printf format spec for pointers
do not cast to unsigned long since a 64-bit pointer
and 32-bit long will truncate the value.

Change-Id: Ibeda98171ccbab4b55950bb02c858773e1028cbf
2011-05-17 12:02:15 -04:00
Asanka C. Herath
59542f4298 Unreachable code to silence older compilers 2010-11-24 15:33:07 -05:00
Love Hornquist Astrand
33b8ccccd6 catch error from as.*printf 2010-05-30 15:13:44 -07:00
Love Hornquist Astrand
e65154c6db catch error from as.*printf 2010-05-30 14:48:48 -07:00