include doc/limitations.rst
include doc/Makefile
include doc/news.rst
+include doc/performance.rst
include doc/pip-requirements.txt
include doc/reference.rst
include doc/thanks.rst
include tests/compli_test_suite/README.md
include tests/compli_test_suite/suite/tc*.ber
include tests/test_compli.py
+include tests/test_crl.py
include tests/test_crts.py
include tests/test_pyderasn.py
include THANKS
+.. _features:
+
Features
========
automatically set required tags)
* Descriptive errors, like ``pyderasn.DecodeError: UTCTime
(tbsCertificate:validity:notAfter:utcTime) (at 328) invalid UTCTime format``
-* ``__slots__``, ``copy.copy()``, ``pickle``, `Cython <https://cython.org/>`__
- friendliness
-* Could be significantly faster and have lower memory usage
- For example parsing of CACert.org's CRL (8.48 MiB) on FreeBSD 12.0
- amd64, Intel Core i5-6200U 2.3 GHz machine, Python 3.5.5/2.7.15:
-
- .. list-table::
- :widths: 15 45 20 20
- :header-rows: 1
-
- * - Library
- - Command
- - Time, sec (Py3/Py2)
- - Memory used, MiB (Py3/Py2)
- * - pyasn1 0.4.5
- - ``der_decode(data, asn1Spec=rfc5280.CertificateList())``
- - 1257 / 1302
- - 1327 / 2093
- * - asn1crypto 0.24.0
- - ``asn1crypto.crl.CertificateList.load(data).native``
- - 29.3 / 43.8
- - 983 / 1677
- * - pyderasn 4.9
- - ``CertificateList().decode(data)`` (CertificateList is
- converted ``pyasn1`` scheme definition)
- - 27.6 / 32.5
- - 498 / 488
+* ``__slots__``, ``copy.copy()`` friendliness
+* Workability with ``pickle``
+* `Cython <https://cython.org/>`__ compatibility
+* Could be significantly :ref:`faster <performance>` and have lower memory usage
* :ref:`Pretty printer <pprinting>` and
:ref:`command-line decoder <cmdline>`, that could
conveniently replace utilities like either ``dumpasn1`` or
:maxdepth: 1
features
+ performance
limitations
examples
reference
* UTCTime and GeneralizedTime allowed values to have plus sign in them,
passing int() check successfully. Prohibit that incorrect behaviour
* UTCTime and GeneralizedTime BER decoding support
-* Faster UTCTime and GeneralizedTime decoding
+* Faster UTCTime and GeneralizedTime decoding, and slightly better
+ overall performance
* Workability under Cython
* Explicitly Check that all ObjectIdentifier arcs are non-negative
--- /dev/null
+.. _performance:
+
+Performance
+===========
+
+.. contents::
+
+Performance is compared between ``pyderasn==6.1``, ``pyasn1==0.4.8`` and
+``asn1crypto==1.3.0``. Decode CACert.org's CRL (2019-02-08 state, 8.72
+MiB), encode it, pickle decoded structure and unpickle it. Machine is
+Intel Core i5-6200U 2.3 GHz, 8 GB RAM, FreeBSD 12.0 amd64, Python 2.7.15
+and Python 3.6.6 from native FreeBSD ports.
+
+Code
+----
+
+pyasn1::
+
+ from pyasn1.codec.der.decoder import decode as der_decoder
+ from pyasn1.codec.der.encoder import encode as der_encoder
+ from pyasn1_modules.rfc5280 import CertificateList
+ with open("revoke.crl", "rb") as fd:
+ raw = fd.read()
+ start = time()
+ crl, _ = der_decoder(raw, asn1Spec=CertificateList())
+ print("decode", time() - start)
+ del raw
+ gc.collect()
+ start = time()
+ der_encoder(c)
+ print("encode", time() - start)
+ # pyasn1 objects are not picklable
+
+asn1crypto::
+
+ from asn1crypto.crl import CertificateList
+ with open("revoke.crl", "rb") as fd:
+ raw = fd.read()
+ start = time()
+ crl = CertificateList.load(raw)
+ c.native # full decoding and Python representation requires that
+ print("decode", time() - start)
+ del raw
+ gc.collect()
+ start = time()
+ crl.dump(force=True) # forced DER encoding
+ print("encode", time() - start)
+ start = time()
+ raw = pickle.dumps(crl)
+ print("dumps", time() - start)
+ del crl
+ gc.collect()
+ print(len(raw))
+ start = time()
+ pickle.loads(raw)
+ print("loads", time() - start)
+
+pyderasn::
+
+ from tests.test_crl import CertificateList
+ with open("revoke.crl", "rb") as fd:
+ raw = fd.read()
+ start = time()
+ crl = CertificateList().decod(raw)
+ print("decode", time() - start)
+ del raw
+ gc.collect()
+ start = time()
+ crl.encode()
+ print("encode", time() - start)
+ start = time()
+ raw = pickle.dumps(crl)
+ print("dumps", time() - start)
+ del crl
+ gc.collect()
+ print(len(raw))
+ start = time()
+ pickle.loads(raw)
+ print("loads", time() - start)
+
+Also there are `cythonized <https://cython.org/>`__ asn1crypto and
+pyderasn versions, made using ``Cython==0.29.14``,
+``FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on
+LLVM 6.0.1)``, ``CFLAGS=-O2`` and Python 3 mode.
+
+DER
+---
+
+.. list-table::
+ :header-rows: 1
+
+ * - Library
+ - Decode time, sec (Py36/Py27)
+ - Encode time, sec (Py36/Py27)
+ - Memory used, MiB (Py36/Py27)
+ * - pyasn1
+ - 1353 / 1400
+ - 37.5 / 36.7
+ - 1645 / 3296
+ * - asn1crypto
+ - 27.5 / 38
+ - 25.7 / 28
+ - 876 / 1742
+ * - cython asn1crypto
+ - 15.6 / N/A
+ - 15.8 / N/A
+ - 880 / N/A
+ * - pyderasn
+ - 33.2 / 33.4
+ - 7.6 / 7.4
+ - 560 / 516
+ * - cython pyderasn
+ - 23 / N/A
+ - 5.9 / N/A
+ - 561 / N/A
+
+asn1crypto performs slightly better (with higher memory cost), but pay
+attention that it contains **much** less data for all objects (like
+offsets, sizes, etc) and it is not strict at all, passing possibly
+invalid DER structures! Also there are plenty of other :ref:`features`
+it lacks.
+
+pickle
+------
+
+pyasn1 objects are not pickable.
+
+.. list-table::
+ :header-rows: 1
+
+ * - Library
+ - dumps time, sec (Py36/Py27)
+ - loads time, sec (Py36/Py27)
+ - Memory used, MiB (Py36/Py27)
+ - Size, MiB (Py36/Py27)
+ * - asn1crypto
+ - 7.9 / 145
+ - 8.3 / 91.4
+ - 2474 / 4944
+ - 174.8 / 373
+ * - pyderasn
+ - 82 / 244
+ - 17 / 77.8
+ - 3010 / 4372
+ - 110.5 / 248.6
--- /dev/null
+# coding: utf-8
+# PyDERASN -- Python ASN.1 DER codec with abstract structures
+# Copyright (C) 2017-2020 Sergey Matveev <stargrave@stargrave.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License as
+# published by the Free Software Foundation, version 3 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this program. If not, see
+# <http://www.gnu.org/licenses/>.
+"""CRL related schemes, just to test the performance with them
+"""
+
+from pyderasn import BitString
+from pyderasn import Sequence
+from pyderasn import SequenceOf
+from pyderasn import tag_ctxc
+
+from tests.test_crts import AlgorithmIdentifier
+from tests.test_crts import CertificateSerialNumber
+from tests.test_crts import Extensions
+from tests.test_crts import Name
+from tests.test_crts import Time
+from tests.test_crts import Version
+
+
+class RevokedCertificate(Sequence):
+ schema = (
+ ("userCertificate", CertificateSerialNumber()),
+ ("revocationDate", Time()),
+ ("crlEntryExtensions", Extensions(optional=True)),
+ )
+
+
+class RevokedCertificates(SequenceOf):
+ schema = RevokedCertificate()
+
+
+class TBSCertList(Sequence):
+ schema = (
+ ("version", Version(optional=True)),
+ ("signature", AlgorithmIdentifier()),
+ ("issuer", Name()),
+ ("thisUpdate", Time()),
+ ("nextUpdate", Time(optional=True)),
+ ("revokedCertificates", RevokedCertificates(optional=True)),
+ ("crlExtensions", Extensions(expl=tag_ctxc(0), optional=True)),
+ )
+
+
+class CertificateList(Sequence):
+ schema = (
+ ("tbsCertList", TBSCertList()),
+ ("signatureAlgorithm", AlgorithmIdentifier()),
+ ("signatureValue", BitString()),
+ )