oysteikt/heimdal
0
0
Fork 0
Code Issues 5 Pull Requests Releases Activity

asn1: Update ASN.1 IOS futures README

Browse Source
This commit is contained in:
Nicolas Williams
2021-02-01 22:28:29 -06:00
parent 6aefc255b6
commit 3b8b9a797c
1 changed files with 124 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

162
lib/asn1/README-X681.md
View File

@@ -1,8 +1,17 @@
Bringing the power of X.682 (ASN.1 Information Object System) to Heimdal Bringing the power of X.682 (ASN.1 Information Object System) to Heimdal
======================================================================== ========================================================================
X.681 is an ITU-T standard in the X.680 series (ASN.1) that is incredibly The base of ASN.1 is specified by X.680, an ITU-T standard.
useful and would be fantastic to implement in Heimdal.
Various extensions are specified in other X.680 series documents:
- X.681: Information Object specification
- X.682: Constraint specification
- X.683: Parameterization of ASN.1 specifications
While X.680 is essential for implementing many Internet (and other) protocols,
implementing a subset of X.681, X.682, and X.683, can enable some magical
features.
This README will cover some ideas for implementation and why we should want This README will cover some ideas for implementation and why we should want
this. This is also covered extensively in RFC 6025, in section 2.1.3. this. This is also covered extensively in RFC 6025, in section 2.1.3.
@@ -10,6 +19,14 @@ this. This is also covered extensively in RFC 6025, in section 2.1.3.
RFC 6025 does an excellent job of elucidating X.681, which otherwise most RFC 6025 does an excellent job of elucidating X.681, which otherwise most
readers unfamiliar with it will no doubt find inscrutable. readers unfamiliar with it will no doubt find inscrutable.
This README should explain the magic that we're after:
Automatic handling of open types by the Heimdal ASN.1 compiler, which,
combined with an implementation of the ASN.1 JSON Encoding Rules (JER
[X.697]), should allow one to build a trivial command-line tool and APIs to
dump as JSON, or parse from JSON, the DER encoding of random PDU types from
random ASN.1 modules, with deep traversal of open types / typed holes.
https://www.itu.int/rec/T-REC-X.681-201508-I/en https://www.itu.int/rec/T-REC-X.681-201508-I/en
@@ -27,9 +44,12 @@ A very common thing to see in projects that use ASN.1, as well as projects that
use alternatives to ASN.1, is a pattern known as the "typed hole" or "open use alternatives to ASN.1, is a pattern known as the "typed hole" or "open
type". type".
The ASN.1 Information Object System (X.681) is all about automating the The ASN.1 Information Object System (IOS) [X.681] is all about automating the
otherwise very annoying task of dealing with "typed holes" / "open types". otherwise very annoying task of dealing with "typed holes" / "open types".
The ASN.1 IOS is not sufficient to implement the magic we're after. Also
needed is constraint specification and parameterization of types.
Typed Holes / Open Types Typed Holes / Open Types
======================== ========================
@@ -73,15 +93,17 @@ or
} }
``` ```
or any number of variations. (Note: the `ANY` variations are no longer or any number of variations.
conformant to X.680 (the base ASN.1 specification).)
Note: the `ANY` variations are no longer conformant to X.680 (the base
ASN.1 specification).
The pattern is `{ id, hole }` where the `hole` is ultimately an opaque sequence The pattern is `{ id, hole }` where the `hole` is ultimately an opaque sequence
of bytes whose content's schema is identified by the `id` in the same data of bytes whose content's schema is identified by the `id` in the same data
structure. structure. The pattern does not require just two fields, and it does not
require any particular type for the hole, nor for the type ID. Sometimes the
Sometimes the "hole" is an `OCTET STRING`, sometimes it's a `BIT STRING`, "hole" is an `OCTET STRING`, sometimes it's a `BIT STRING`, sometimes it's an
sometimes it's an `ANY` or `ANY DEFINED BY`. `ANY` or `ANY DEFINED BY`.
An example from PKIX: An example from PKIX:
@@ -199,8 +221,8 @@ display or compile DER (and other encodings) of certifcates and many other
interesting data structures. interesting data structures.
ASN.1 IOS ASN.1 IOS, Constraint, and Parameterization
========= ===========================================
The ASN.1 IOS is additional syntax that allows ASN.1 module authors to express The ASN.1 IOS is additional syntax that allows ASN.1 module authors to express
all the details about typed holes that ASN.1 compilers need to make developers' all the details about typed holes that ASN.1 compilers need to make developers'
@@ -210,40 +232,72 @@ RFC5912 has lots of examples, such as this `CLASS` corresponding to the
`Extension` type from PKIX: `Extension` type from PKIX:
``` ```
-- A class that provides some of the details of the PKIX Extension typed
-- hole:
EXTENSION ::= CLASS { EXTENSION ::= CLASS {
&id OBJECT IDENTIFIER UNIQUE, -- The following are fields of a class (as opposed to "members" of
&ExtnType, -- SEQUENCE or SET types):
&Critical BOOLEAN DEFAULT {TRUE | FALSE } &id OBJECT IDENTIFIER UNIQUE, -- UNIQUE -> this is the hole type ID.
&ExtnType, -- This is a type field (the hole).
&Critical BOOLEAN DEFAULT {TRUE | FALSE } -- this is a value field.
} WITH SYNTAX { } WITH SYNTAX {
-- This is a specification of easy to use (but hard-to-parse) syntax for
-- specifying instances of this CLASS:
SYNTAX &ExtnType IDENTIFIED BY &id SYNTAX &ExtnType IDENTIFIED BY &id
[CRITICALITY &Critical] [CRITICALITY &Critical]
} }
Extensions{EXTENSION:ExtensionSet} ::= -- Here's a parameterized Extension type. The formal parameter is an as-yet
SEQUENCE SIZE (1..MAX) OF Extension{{ExtensionSet}} -- unspecified set of valid things this hole can carry for some particular
-- instance of this type. The actual parameter will be specified later (see
-- below).
Extension{EXTENSION:ExtensionSet} ::= SEQUENCE { Extension{EXTENSION:ExtensionSet} ::= SEQUENCE {
-- The type ID has to be the &id field of the EXTENSION CLASS of the
-- ExtensionSet object set parameter.
extnID EXTENSION.&id({ExtensionSet}), extnID EXTENSION.&id({ExtensionSet}),
-- This is the critical field, whose DEFAULT value should be that of
-- the &Critical field of the EXTENSION CLASS of the ExtensionSet object
-- set parameter.
critical BOOLEAN critical BOOLEAN
-- (EXTENSION.&Critical({ExtensionSet}{@extnID})) -- (EXTENSION.&Critical({ExtensionSet}{@extnID}))
DEFAULT FALSE, DEFAULT FALSE,
-- Finally, the hole is an OCTET STRING constrained to hold the encoding
-- of the type named by the &ExtnType field of the EXTENSION CLASS of the
-- ExtensionSet object set parameter.
--
-- Note that for all members of this SEQUENCE, the fields of the object
-- referenced must be of the same object in the ExtensionSet object set
-- parameter. That's how we get to say that some OID implies some type
-- for the hole.
extnValue OCTET STRING (CONTAINING extnValue OCTET STRING (CONTAINING
EXTENSION.&ExtnType({ExtensionSet}{@extnID})) EXTENSION.&ExtnType({ExtensionSet}{@extnID}))
-- contains the DER encoding of the ASN.1 value -- contains the DER encoding of the ASN.1 value
-- corresponding to the extension type identified -- corresponding to the extension type identified
-- by extnID -- by extnID
} }
-- This is just a SEQUENCE of Extensions, the parameterized version.
Extensions{EXTENSION:ExtensionSet} ::=
SEQUENCE SIZE (1..MAX) OF Extension{{ExtensionSet}}
``` ```
and these uses of it in RFC5280 (PKIX base): and these uses of it in RFC5280 (PKIX base):
``` ```
-- Here we have an individual "object" specifying that the OID
-- id-ce-authorityKeyIdentifier implies AuthorityKeyIdentifier as the hole
-- type:
ext-AuthorityKeyIdentifier EXTENSION ::= { SYNTAX ext-AuthorityKeyIdentifier EXTENSION ::= { SYNTAX
AuthorityKeyIdentifier IDENTIFIED BY AuthorityKeyIdentifier IDENTIFIED BY
id-ce-authorityKeyIdentifier } id-ce-authorityKeyIdentifier }
-- And here's the OID, for completeness:
id-ce-authorityKeyIdentifier OBJECT IDENTIFIER ::= { id-ce 35 } id-ce-authorityKeyIdentifier OBJECT IDENTIFIER ::= { id-ce 35 }
... ...
-- And Here's an object set for the EXTENSION CLASS collecting a bunch of
-- related extensions (here they are the extensions that certificates can
-- carry in their extensions member):
CertExtensions EXTENSION ::= { CertExtensions EXTENSION ::= {
ext-AuthorityKeyIdentifier | ext-SubjectKeyIdentifier | ext-AuthorityKeyIdentifier | ext-SubjectKeyIdentifier |
ext-KeyUsage | ext-PrivateKeyUsagePeriod | ext-KeyUsage | ext-PrivateKeyUsagePeriod |
@@ -257,6 +311,11 @@ and these uses of it in RFC5280 (PKIX base):
ext-SubjectInfoAccessSyntax, ... } ext-SubjectInfoAccessSyntax, ... }
... ...
-- Lastly, we have a Certificate, and the place where the Extensions type's
-- actual parameter is specified.
--
-- This is where the *rubber meets the road*!
Certificate ::= SIGNED{TBSCertificate} Certificate ::= SIGNED{TBSCertificate}
TBSCertificate ::= SEQUENCE { TBSCertificate ::= SEQUENCE {
@@ -275,11 +334,17 @@ and these uses of it in RFC5280 (PKIX base):
]], ]],
[[3: -- If present, version MUST be v3 -- [[3: -- If present, version MUST be v3 --
extensions [3] Extensions{{CertExtensions}} OPTIONAL extensions [3] Extensions{{CertExtensions}} OPTIONAL
-- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-- The rubber meets the road *here*.
--
-- This says that the set of *known* certificate
-- extensions are those for which there are "objects"
-- in the "object set" named CertExtensions.
]], ... } ]], ... }
``` ```
Notice that the `extensions` field of `TBSCertificate` is of type `Extensions` Notice that the `extensions` field of `TBSCertificate` is of type `Extensions`
parametrized by the `CertExtensions` IOS object set. parametrized by the `CertExtensions` information object set.
This allows the compiler to know that if any of the OIDs listed in the This allows the compiler to know that if any of the OIDs listed in the
`CertExtensions` object set appear as the actual value of the `extnID` member `CertExtensions` object set appear as the actual value of the `extnID` member
@@ -292,20 +357,20 @@ an `Extension` with `extnID == id-ce-authorityKeyIdentifier` must have an
Implementation Thoughts Implementation Thoughts
======================= =======================
- The ASN.1 IOS is fairly large and non-trivial. Perhaps we can just bake in - The required specifications, X.681, X.682, and X.683, are fairly large and
a few useful IOS classes without adding support for defining arbitrary non-trivial. Perhaps we can implement just the subset of those three that
classes. we need to implement PKIX, just as we already implement just the subset of
X.680 that we need to implement PKIX and Kerberos.
For dealing with PKIX, the bare minimum of IOS classes we should want are: For dealing with PKIX, the bare minimum of IOS classes we should want are:
- ATTRIBUTE (used for DN attributes in PKIX base) - `ATTRIBUTE` (used for `DN` attributes in RFC5280)
- EXTENSION (used for certificate attributes in PKIX base) - `EXTENSION` (used for certificate extensions in RFC5280)
- `TYPE-IDENTIFIER` (used for CMS)
Then we can implement support for just declarations of information objects The minimal subset of X.681, X.682, and X.683 needed to implement those
and information object sets in `lib/asn1parse.y`, which is probably not a three is all we need. Eventually we may want to increase that subset so as
very big deal. to implement other IOS classes from PKIX, such as `DIGEST-ALGORITHM`
Internally we can have a function for creating a class.
- We'll really want to do this mainly for the template compiler and begin - We'll really want to do this mainly for the template compiler and begin
abandoning the original compiler -- hacking on two compilers is difficult, abandoning the original compiler -- hacking on two compilers is difficult,
@@ -314,17 +379,21 @@ Implementation Thoughts
rules supported and `N` is the number of types in an ASN.1 module (or all rules supported and `N` is the number of types in an ASN.1 module (or all
modules). modules).
- Also, to make the transition to using IOS in-tree, we'll want to add fields - Also, to make the transition to using IOS in-tree, we'll want to keep
to the C structures generated by the compiler today, that way code that existing fields of C structures as generated by the compiler today, only
hasn't been updated to use the automatic encoding/decoding can still work. adding new ones, that way code that hasn't been updated to use the automatic
encoding/decoding can still work and we can then update Heimdal in-tree
slowly to take advantage of the new magic.
Thus `Extension` should compile to: Thus `Extension` should compile to:
``` ```
typedef struct Extension { typedef struct Extension {
-- Existing fields:
heim_oid extnID; heim_oid extnID;
int *critical; int *critical;
heim_octet_string extnValue; heim_octet_string extnValue;
-- New, CHOICE-like fields:
enum Extension_iosnum { enum Extension_iosnum {
Extension_iosnumunknown = 0, /* when the extnID is unrecognized */ Extension_iosnumunknown = 0, /* when the extnID is unrecognized */
Extension_iosnum_ext_AuthorityKeyIdentifier = 1, Extension_iosnum_ext_AuthorityKeyIdentifier = 1,
@@ -343,11 +412,28 @@ Implementation Thoughts
If a caller to `encode_Certificate()` passes a certificate object with If a caller to `encode_Certificate()` passes a certificate object with
extensions with `_ioselement == Extension_iosnumunknown`, then the encoder extensions with `_ioselement == Extension_iosnumunknown`, then the encoder
should use the `extnID` and `extnValue` fields, otherwise it should use the should use the `extnID` and `extnValue` fields, otherwise it should use the
`_ioselement` and `_iosu` fields. (In both cases, the `critical` field `_ioselement` and `_iosu` fields and ignore the `extnID` and `extnValue`
should get used.) fields.
- We'll need to reduce the number of bits used to encode tag values in the In both cases, the `critical` field should get used as-is. The rule should
templates. Currently we use 20 bits, but that's far too many. We can be that we support *two* special fields: a hole type ID enum field, and a
almost certainly get away with using only 10 bits for tags. This will allow decoded hole value union. All other fields will map to either normal
us to have more opcodes, which we'll need more of in order to handle typed members of the SET/SEQUENCE, or to members that are derived from a CLASS but
holes described by IOS classes and information object sets. which are neither hole type ID fields nor hole fields.
- Type ID values must get mapped to discrete enum values. We'll want type IDs
to be sorted, too, so that we can binary search the "object set" when
decoding. For encoding we'll want to "switch" on the mapped type ID enum.
- The ASN.1 parser merely builds an AST. That will not change.
- The C header generator will remain shared between the two backends.
- Only the template backend will support the ASN.1 IOS. We'll basically
encode a new template for the combination of object set and typed hole
container type. This will come with a header entry indicating how many
items are in the object set, and each item will be one entry pointing to the
template for one particular object in the object set. The template for each
object will identify the type ID and the template for the associated type.
Perhaps we'll inline the objects for locality of reference.