chiark / gitweb /
Mark Wooding [Sat, 11 Apr 2020 14:30:26 +0000 (15:30 +0100)]
@@@ mdwopt buggy wip
Mark Wooding [Sat, 28 Mar 2020 10:37:20 +0000 (10:37 +0000)]
@@@ mdwopt broken wip
Mark Wooding [Sat, 28 Mar 2020 10:34:57 +0000 (10:34 +0000)]
@@@ mLib-python Pyke wip
Mark Wooding [Sat, 28 Mar 2020 10:00:09 +0000 (10:00 +0000)]
pyke/mapping.c: Export `gmap_pyupdate' as a separately useful thing.
It adds all of the things in a Python mapping into a gmappish object.
What's not to like?
Mark Wooding [Wed, 27 Nov 2019 21:04:53 +0000 (21:04 +0000)]
debian/rules: Build using the provided Makefile.
There were several problems with the old system. Firstly, it only
actually worked by accident: setting an explicit build directory caused
Debhelper to search there for a makefile (which it didn't find, for
obvious reasons) and fall back to the Python distutils system. That
would be OK, but only because there's only been one Python version I've
wanted to support in Debian for ages; but that's likely to change. And
it won't run tests properly.
Instead, delete the `-B' option (which will break the upcoming tests).
Now Debhelper will build via the Makefile. Set the `PYTHONS'
environment variable to the Python versions supported in Debian. And
set the prefix correctly during installation.
Mark Wooding [Wed, 27 Nov 2019 21:03:51 +0000 (21:03 +0000)]
debian/: Bump to Debhelper 10.
Mark Wooding [Sat, 11 Apr 2020 14:22:40 +0000 (15:22 +0100)]
Mark Wooding [Sat, 23 Nov 2019 15:05:39 +0000 (15:05 +0000)]
pyke/pyke-mLib.c: Raise `OverflowError' on out-of-range inputs.
This seems to match better what Python does under these conditions.
Mark Wooding [Tue, 22 Oct 2019 18:12:28 +0000 (19:12 +0100)]
Port to Python 3.
Most of this is `#ifdef ...' ... `#endif' noise, with a few tweaks
thrown in.
Some notes on specific parts of the port.
* buffer.c: The Python 3 buffer protocol is completely different.
Read buffers work differently, but aren't problematic; write buffers
can now be held open for an extended period, so we need the locking
machinery that was added recently.
* catacomb.c: Module initialization has changed, but isn't a great
deal more difficult.
* catacomb/__init__.py, catacomb/pwsafe.py, pock, pwsafe: There are a
number of places which need language-version switches, but none of
them is especially interesting. This diff is noisier than it should
be because I couldn't adjust the indentation in advance.
* mp.c: With the abolition of a separate fixnum type, `mp_frompylong'
needed to express the fast path from a fixnum in a different way.
* pwsafe: The hacking to alter the error-handling behaviour associated
with the `stdout' stream is deeply unpleasant. Sorry.
* pyke/pyke.h: Most of the porting work happens here, with alternative
definitions for the various macros introduced earlier.
* .gitignore: Ignore Python 3 `__pycache__/' turds. Python 3 leaves
its pre-tokenized files in `__pycache__/' directories, which somehow
manage to be much more objectionable than the loose Python 2 `*.pyc'
files. Ignore these.
* debian/: Add the necessary things to build a Python 3 extension
package.
Mark Wooding [Tue, 22 Oct 2019 17:50:19 +0000 (18:50 +0100)]
*.[ch]: Some preparatory reformatting for the Python 3 porting.
Nothing very complicated or profound. These are intended to make the
upcoming diff easier to check for stupid mistakes. A lot of this is
whitespace change that disappears in `git show -w'.
Mark Wooding [Tue, 22 Oct 2019 17:31:57 +0000 (18:31 +0100)]
mp.c, catacomb/__init__.py, pyke/: Fix mixed-mode arithmetic involving `float'.
This is a bit embarrassing.
>>> import catacomb as C
>>> x = C.MP(5)
>>> x == 5.1
True
>>> x < 5.1
False
>>> r = x/2
>>> r
5/2
>>> r == 2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: an integer is required
>>> r == 2.5
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: an integer is required
>>> r*1.0
5/2
>>> r*1.1
5/2
>>> r*2.0
5
>>> r*2.5
5
Fix this nonsense.
* Change the `obvious' arithmetic operators so that they notice that
one of the operands is a float. Handle this by converting to a
Python bignum and letting Python handle the arithmetic. The result
is a float, which seems sensible inexact contagion.
* Introduce a rich-comparison method which also detects a float
operand and hands off to Python. Python seems to get this right,
comparing the float to the bignum in its full precision, so that's a
win.
* Also, modify the `IntRat' code to apply inexact contagion in the
same way. Comparisons may be imperfect here, but that's
surprisingly hard to get right.
The new results:
>>> import catacomb as C
>>> x = C.MP(5)
>>> x == 5.1
False
>>> x < 5.1
True
>>> r = x/2
>>> r
5/2
>>> r == 2
False
>>> r == 2.5
True
>>> r*1.0
2.5
>>> r*1.1
2.75
>>> r*2.0
5.0
>>> r*2.5
6.25
Mark Wooding [Mon, 21 Oct 2019 01:38:59 +0000 (02:38 +0100)]
*.c: Separate string function calls according to text/binary usage.
Don't use the `PyString_...' functions directly. Instead, define two
collections of macros `TEXT_...' and `BIN_...', which can be used on
text and binary strings respectively. The two sets are slightly
different, because they've been designed to adapt to Python 3's
different interfaces.
There are no `PyString...' calls remaining outside of these macros.
Mark Wooding [Mon, 21 Oct 2019 00:43:54 +0000 (01:43 +0100)]
*.c: Introduce a new input conversion for binary strings.
One of the major differences in Python 3 is that it firmly distinguishes
between binary and text strings: the former consist of small integers,
while the latter consist of Unicode scalars. The Python 3 `s#'
conversion's main purpose is to accept text strings, and though it will
also accept binary strings it's not really ideal for the purpose.
Python 3 introduces a new conversion `y#' specifically for binary
strings, though this isn't quite what we want because, for some reason,
it /doesn't/ work with bufferish objects which require explicit release.
The best answer seems to be to introduce our own custom conversion for
binary strings, so we do this here, replacing all of the binary-input
argument conversions. While we're at it, replace all of the by-steam
argument conversions using `PyObject_AsReadBuffer' too.
Mark Wooding [Sun, 20 Oct 2019 23:33:22 +0000 (00:33 +0100)]
*.c: Use the new `Py_hash_t' type.
Mark Wooding [Sun, 20 Oct 2019 23:14:01 +0000 (00:14 +0100)]
*.c: Use `PyVarObject_HEAD_INIT' to initialize type object headers.
Define a suitable version if the Python headers don't do it for us.
Mark Wooding [Sun, 20 Oct 2019 23:19:44 +0000 (00:19 +0100)]
*.c: Use the new `Py_TYPE' and `Py_SIZE' macros; define them if necessary.
Python 2.6 introduced macros to access object header fields, which is
handy because Python 3 changes the way object headers are
structured. (I think the binary layout is unchanged, but the C-level
structuring is definitely different.)
This is the start of Python 3 porting work.
Mark Wooding [Mon, 18 Nov 2019 10:41:12 +0000 (10:41 +0000)]
pyke/mapping.c, key.c: Make the mapping code more intrusive and complete.
Previously, mapping classes would implement a minimum of Python-level
operations, such as iterating over keys and mapping lookups, and the
generic mapping code would just synthesize missing operations in terms
of these.
Instead, introduce a mapping-specific vtable pointer in object
structures which describes the mapping operations in a form similar to
mLib's `sym' interface, which is rather more efficient in terms of hash
probes, and have the generic mapping code synthesize all of the Python-
level operations from these.
Needless to say, this is very messy. Sorry.
Mark Wooding [Sun, 20 Oct 2019 21:57:54 +0000 (22:57 +0100)]
pyke/mapping.c: Introduce macro for unconstifying common keyword list.
Mark Wooding [Tue, 22 Oct 2019 17:53:50 +0000 (18:53 +0100)]
pyke/pyke.c (newtype): Explicitly clear `ht_slots'.
This makes things go very badly wrong in Python 3. I'm not sure why it
doesn't go wrong in Python 2...
Mark Wooding [Sun, 20 Oct 2019 22:18:09 +0000 (23:18 +0100)]
*.c: Split the constant definitions into the various submodules.
Mark Wooding [Sun, 20 Oct 2019 20:18:08 +0000 (21:18 +0100)]
pyke/pyke.[ch]: Make type skeleton structures be read-only.
We couldn't do this before because `INITTYPE_META' would write the
direct superclass to the `tp_base' slot in the skeleton before calling
`inittype'. To make this work, then, we adjust `inittype' to take the
direct superclass as an extra argument and plug it into the newly
created heap-type; and `INITTYPE_META' needs adjusting to pass this
argument rather than trying to write to the skeleton directly.
Of course, then we need to actually mark the type skeletons as `const'.
Mark Wooding [Sun, 20 Oct 2019 11:43:40 +0000 (12:43 +0100)]
pyke/pyke.h: Add a `MEMBER' variant with explicit member name.
Mark Wooding [Sun, 20 Oct 2019 20:00:18 +0000 (21:00 +0100)]
*.c: Use Python `METH_NOARGS' methods where applicable.
Rather than rolling our own. We need a little extra machinery to insert
the table entries, especially for the generic mapping support, but this
still saves 95 lines of code.
The `CONVERT_CAREFULLY' macro recently added will check that we haven't
messed things up too badly.
Mark Wooding [Tue, 19 Nov 2019 00:19:38 +0000 (00:19 +0000)]
*.c: Use Python's facilities for defining class and static methods.
These weren't available when this library was first written, but the
workaround is rather ugly and no longer necessary.
Mark Wooding [Sun, 20 Oct 2019 21:46:05 +0000 (22:46 +0100)]
*.c: Make all of the type-definition tables read-only.
The hard part is a new collection of macros which strip the `const'
qualifier from the actual table definitions, after checking that they
actually have the correct type.
And then there's the slog of actually changing all of the definitions
and using the new macros.
Mark Wooding [Sun, 20 Oct 2019 18:24:37 +0000 (19:24 +0100)]
pyke/pyke.c: Check conversions hidden inside `KWLIST' and `KWMETH'.
Introduce a new macro `CONVERT_CAREFULLY' which checks (via a rather
sleazy hack) that its operand is of the expected type before converting
it to some other type. This is protected by an `#ifdef' guard because
I'm thinking about adding it to a version of mLib, but I want to keep
the Pyke core independent of mLib.
Use this new macro to build better versions of `KWLIST' and `KWMETH',
which have casts hidden inside them, so that we can now be certain that
the method table entries match up with the functions. (Spoiler: they
didn't, quite, and commit
19bff42f99d41cd9b8953ff61edfe35380b54d88,
backported onto the 1.1.x branch, fixes the bug that was found by this
change.)
Mark Wooding [Sun, 20 Oct 2019 18:17:41 +0000 (19:17 +0100)]
pyke/pyke.h, key.c: Rename `INDEXERR' to `MAPERR'.
The name `INDEXERR' was pointlessly confusing, given the existing
`IXERR' which actually raises Python's `IndexError'. It's important,
though, that `KEYERR' be reserved for Catacomb's `KeyError' exception,
which really is an error about (cryptographic) keys. So `MAPERR' it is.
Mark Wooding [Sun, 20 Oct 2019 16:51:13 +0000 (17:51 +0100)]
pyke/, ...: Extract utilities into a sort-of reusable library.
This commit changes no code, but it moves a lot of it about. Tidying up
will come later.
Mark Wooding [Fri, 10 Apr 2020 22:02:06 +0000 (23:02 +0100)]
(Pruned history of `catacomb-python' begins here.)
The code which is now the `Pyke' library used to be part of the Catacomb
Python bindings. Beginning here is a pruned version of that history,
limited to the files which actually contributed to the `Pyke' codebase.
It was constructed from the original history, starting at revision
10e6f88a407ce04807b6ad76a81be553f8d7abaa, by running
git filter-branch --prune-empty --index-filter '
git ls-files -s | {
mode=old
keep=
while read m h s f; do
case $mode,$f in
old,util.c | old,catacomb.c | old,catacomb-python.h)
keep="$keep $m,$h,$f" ;;
old,pyke/*)
keep="$m,$h,${f#*/}" mode=new ;;
new,pyke/*)
keep="$keep $m,$h,${f#*/}" ;;
esac
git update-index --force-remove $f
done
for i in $keep; do
git update-index --add --cacheinfo $i
done
}
' pyke-prehistory.head -- pyke/ util.c catacomb.c catacomb-python.h
and the tucking this empty commit beneath the head which actually
introduces the `pyke/' directory. (This subterfuge is, unfortunately,
necessary for `git subtree' to construct the right history.)
Mark Wooding [Tue, 22 Oct 2019 18:22:07 +0000 (19:22 +0100)]
catacomb-python.h: Delete a stray trailing `\'.
This doesn't actually hurt anything because of the following blank line,
but, umm, ...
Mark Wooding [Sun, 20 Oct 2019 17:18:05 +0000 (18:18 +0100)]
catacomb-python.h: Add a macro to declare module init functions.
Mark Wooding [Sun, 20 Oct 2019 17:56:07 +0000 (18:56 +0100)]
util.c: Replace mLib `DISCARD' with a plain `(void)' cast.
Mark Wooding [Sun, 20 Oct 2019 17:03:35 +0000 (18:03 +0100)]
util.c: Rewrite `addmethods' to remove dependency on <mLib/darray.h>.
Mark Wooding [Sun, 20 Oct 2019 20:03:38 +0000 (21:03 +0100)]
catacomb.c: Use a less awful version comparison.
Mark Wooding [Sun, 20 Oct 2019 16:57:11 +0000 (17:57 +0100)]
catacomb.c, util.c: Export `modname' and set it in main entry point.
This is preliminary work in splitting out the Catacomb-independent
utilities so that they can be used by other projects.
Mark Wooding [Sat, 19 Oct 2019 13:31:40 +0000 (14:31 +0100)]
*.c: Use `Py_XDECREF' where applicable.
The Python 2.5 version of `Py_DECREF' isn't properly braced, so `if (x)
Py_DECREF(x);' provokes a dangling-`else' warning from the compiler.
Mark Wooding [Fri, 22 Nov 2019 20:30:31 +0000 (20:30 +0000)]
*.c: Reformat docstrings.
No functional changes here: just changing how the strings are
represented in the source.
Mark Wooding [Fri, 22 Nov 2019 22:43:02 +0000 (22:43 +0000)]
catacomb-python.h: Give up on Python versions prior to 2.5.
Mark Wooding [Sun, 24 Nov 2019 12:35:45 +0000 (12:35 +0000)]
*.c: Use Python macros rather than functions where possible.
Mark Wooding [Sun, 20 Oct 2019 14:56:54 +0000 (15:56 +0100)]
catacomb-python.h, *.c: Move definitions back into implementation files.
The header was chock full of implementation details which weren't
needed by anything. Move things which aren't needed further afield back
into their home files, and delete things like conversion functions which
aren't actually being used.
Also, some light reordering of the things that are left.
(This patch turns out to be remarkably commutative through the enormous
pile of changes coming up.)
Mark Wooding [Wed, 27 Nov 2019 15:12:23 +0000 (15:12 +0000)]
Merge branch '1.3.x'
* 1.3.x: (101 commits)
rand.c: Show keyword argument as optional.
mp.c: Fix punctuation error in docstrings.
t/t-*.py: Use the `WriteBuffer.contents' property.
t/t-bytes.py: Check that indexing, slicing, etc. return `C.ByteString'.
t/t-algorithms.py: Add a simple test for `Keccak1600.copy'.
t/t-algorithms.py: Add tests for other HSalsa20 and HChaCha key sizes.
t/t-algorithms.py: Add AEAD tests.
t/t-algorithms.py: Add tests for the new `KeySZ.pad' method.
catacomb/__init__.py (KeySZRange.pad): Return correct value.
algorithms.c: Propagate `AEADF_NOAAD' to `aad' objects.
algorithms.c (
AEADAAD.copy): Propagate the hashed length to the copy.
t/: Add a test suite.
ec.c: Don't lose error status when constructing points from a sequence.
ec.c: Free partially constructed points coordinatewise.
*.c: Be more careful about `PySequence_Size'.
key.c: Reformat the rest of the `KeyError' constructor.
key.c: Parse `KeyError' constructor arguments by hand.
catacomb-python.h: Add a macro for raising `OverflowError'.
key.c: Collect `KeyError' argument count as a separate step.
key.c: Use tuple functions on `KeyError' argument tuple.
...
Mark Wooding [Wed, 27 Nov 2019 15:11:08 +0000 (15:11 +0000)]
Merge branch '1.2.x' into 1.3.x
* 1.2.x: (89 commits)
t/: Add a test suite.
ec.c: Don't lose error status when constructing points from a sequence.
ec.c: Free partially constructed points coordinatewise.
*.c: Be more careful about `PySequence_Size'.
key.c: Reformat the rest of the `KeyError' constructor.
key.c: Parse `KeyError' constructor arguments by hand.
catacomb-python.h: Add a macro for raising `OverflowError'.
key.c: Collect `KeyError' argument count as a separate step.
key.c: Use tuple functions on `KeyError' argument tuple.
key.c: Rename sad-path label to `end'.
key.c: Delete duplicate setting of `errstring'.
util.c (mkexc): Populate dictionary before constructing exception class.
key.c: Only set the error code.
catacomb.c, util.c: Publish negative constants correctly.
field.c: Delete the completely unused `getfe' function.
key.c (convfilter): Fix sense of error tests.
buffer.c, ec.c: Fix required size for EC `buffer' encoding.
algorithms.c: Fix `max' property name in docstrings.
catacomb/__init__.py (_HashBase): Check that integers are within bounds.
debian/rules: Build using the provided Makefile.
...
Mark Wooding [Sun, 24 Nov 2019 16:36:24 +0000 (16:36 +0000)]
*.c: Be more careful about `PySequence_Size'.
This can be implemented by Python, so it can throw exceptions.
Fortunately, Python checks that the result is nonnegative, so we don't
have to worry about that.
Mark Wooding [Sun, 24 Nov 2019 14:59:35 +0000 (14:59 +0000)]
catacomb-python.h: Add a macro for raising `OverflowError'.
Mark Wooding [Sun, 24 Nov 2019 15:07:08 +0000 (15:07 +0000)]
util.c (mkexc): Populate dictionary before constructing exception class.
It turns out that the dictionary contents are copied into the new class
rather than being used by reference, so populating it later doesn't
actually do anything.
Mark Wooding [Sun, 24 Nov 2019 15:11:46 +0000 (15:11 +0000)]
catacomb.c, util.c: Publish negative constants correctly.
The various `KERR_...' constants and `PGEN_ABORT' are canonically
negative integers, but weren't published correctly. Add a flags word to
`struct nameval' to identify constants which should really be signed,
and some fancy footwork to convert unsigned integers back to negative
values without upsetting C. Adjust the `C' macro to spot negative
constants and mark them specially.
Not all is well. In particular, the `KEXP_...' constants should /not/
be published as signed values, even if `time_t' is signed on the target
platform, because the Python bindings handle them exclusively as
`unsigned long' values. To make this work, also introduce `CF' which
allows us to set the flags explicitly, and use it for `KEXP_...'.
Mark Wooding [Sat, 23 Nov 2019 16:04:29 +0000 (16:04 +0000)]
field.c: Delete the completely unused `getfe' function.
It was introduced in the original commit, and never used at all.
Mark Wooding [Sun, 17 Nov 2019 03:14:31 +0000 (03:14 +0000)]
catacomb.c: Publish `RAND_IBITS' constant.
This is a limit on how much you can request in `seed', so it's sort of
important.
Mark Wooding [Sun, 17 Nov 2019 01:34:12 +0000 (01:34 +0000)]
util.c: Make `default' arguments optional in `get', `setdefault', `pop'.
Alas, `pop' is a little trickier than the others.
Mark Wooding [Sun, 17 Nov 2019 01:33:34 +0000 (01:33 +0000)]
util.c: Fix sense of error check in `popitem'.
Mark Wooding [Thu, 14 Nov 2019 19:08:53 +0000 (19:08 +0000)]
catacomb-python.h, *.c: Fix how Python `pgen' handlers handle exceptions.
Oh, this was a mess. The old code would convert an exception from a
Python handler into `PGEN_ABORT', and hope that the exception state was
still available when the overall operation ended.
This doesn't work. In particular, steppers and testers are finalized by
calling them with `PGEN_DONE', and the interpreter doesn't like
re-entering Python with an exception set. (In debug builds, this is an
assertion failure.)
Overhaul all of this nonsense.
* Add a collection of utilities for saving and restoring the exception
state.
* Add a hook, in the `catacomb' module, for reporting `lost'
exceptions, for the case where further exceptions are raised while
responding to a first exception.
* Use a larger `pypgev' structure to track the state of a Python event
handler through the framework. This structure holds a reference to
the Python object itself, and a slot for recording an exception.
* When a Python handler fails, stash the exception state in the slot
provided by the `pypgev' structure if there isn't one already, and
clear the pending exception. If there is already an exception in
the slot, then report the new exception through the hook described
above.
* Once a `pgen' operation completes, if it raised any exceptions at
all, then the first of these is left in the exception slot. If it
fails otherwise, then we supply a generic exception.
Mark Wooding [Sun, 10 Nov 2019 22:39:15 +0000 (22:39 +0000)]
mp.c, util.c: Use `Py_ssize_t' for lengths.
Missed some from the earlier pass.
Mark Wooding [Sat, 19 Oct 2019 19:33:28 +0000 (20:33 +0100)]
util.c: Fix docstrings for generic-map iterator classes.
Mark Wooding [Tue, 15 Oct 2019 10:48:12 +0000 (11:48 +0100)]
mp.c: Factor out and export `mphash'.
Mark Wooding [Sat, 19 Oct 2019 19:02:08 +0000 (20:02 +0100)]
util.c: Use Python's machinery for handling 64-bit integers.
Where available.
Mark Wooding [Sat, 19 Oct 2019 18:59:59 +0000 (19:59 +0100)]
util.c: Augment `convu64' to collect its argument via `kludge64'.
Otherwise integers too large for `unsigned long' get rejected with an
error, even though they ought to be acceptable as a `uint64'.
Mark Wooding [Fri, 4 Oct 2019 14:40:28 +0000 (15:40 +0100)]
Deploy the new <ctype.h> and `foocmp' macros from mLib.
Mark Wooding [Tue, 1 Oct 2019 10:40:41 +0000 (11:40 +0100)]
Set ELF visibility to keep all of our external symbols to ourselves.
Mark Wooding [Fri, 9 Nov 2018 15:35:14 +0000 (15:35 +0000)]
algorithms.c, etc.: Support the new AEAD abstraction.
The new machinery means we can reimplement `secret_box' and
`secret_unbox' using Catacomb's `..._naclbox' AE scheme.
Mark Wooding [Thu, 6 Dec 2018 11:34:50 +0000 (11:34 +0000)]
debian/control: Add Build-Depends for `dh-python'.
Because it asked me so nicely.
Mark Wooding [Fri, 9 Nov 2018 12:22:16 +0000 (12:22 +0000)]
algorithms.c, knock-on: Eliminate `f_freeme' flags.
These were being carefully set and checked on ciphers, hashes, and MACs,
for no good reason because the flag was always set. Abolish it.
The flag still exists for `grand' objects because they get injected into
Python through the `pgen' event-handling machinery, and Python mustn't
destroy them just because it loses its reference. (It also mustn't try
to hang onto them, so there's a longstanding bug in there.)
Mark Wooding [Fri, 9 Nov 2018 12:28:16 +0000 (12:28 +0000)]
bytestring.c: Implement indexing, slicing, concatenation and repeating.
Now these return bytestring objects, rather than Python strings.
Mark Wooding [Fri, 9 Nov 2018 12:16:59 +0000 (12:16 +0000)]
Consistently make keyword-lists be static and read-only.
We had an exciting mix of static and automatic storage durations, and
none were marked as `const'. Python isn't `const'-correct, so we have
to cast away the `const'-ness: introduce a new macro `KWLIST' to do
this.
Also constify some other related tables, such as method names in
`pgev_python'.
Mark Wooding [Fri, 9 Nov 2018 12:34:20 +0000 (12:34 +0000)]
catacomb-python.h: Eliminate redundant declaration of `convgcipher'.
There's another one a bit further down.
Mark Wooding [Sun, 10 Sep 2017 10:31:21 +0000 (11:31 +0100)]
Release 1.1.1.
Mark Wooding [Sat, 9 Sep 2017 22:17:47 +0000 (23:17 +0100)]
utils.pyx (_getfd): Hack around Pyrex exception-handling bugs.
It seems that Pyrex has some nasty bugs here. Most obviously, it fails
to cancel the exception when handling it. But even then, there's a
code-generation bug around returning in an `except' block which leads to
a null-pointer dereference.
These are Debian #875284 and #875285.
Mark Wooding [Sat, 9 Sep 2017 22:11:09 +0000 (23:11 +0100)]
fdutils.pyx (fdflags): Fix disastrous variable-name error.
Mark Wooding [Wed, 16 Aug 2017 13:28:59 +0000 (14:28 +0100)]
Release 1.1.0.
Mark Wooding [Wed, 16 Aug 2017 02:08:26 +0000 (03:08 +0100)]
debian/control: Build-depend on `python-all-dev', not explicit versions.
Mark Wooding [Wed, 16 Aug 2017 02:07:36 +0000 (03:07 +0100)]
debian/: Use `dh_python2' for packaging.
Mark Wooding [Wed, 16 Aug 2017 01:54:57 +0000 (02:54 +0100)]
debian/copyright, .mailmap: Convert to machine-readable format.
Mark Wooding [Fri, 28 Jul 2017 23:57:03 +0000 (00:57 +0100)]
*.pyx: Add some rather laconic docstrings.
Mark Wooding [Fri, 28 Jul 2017 23:56:17 +0000 (00:56 +0100)]
codex.pyx.in: Zap trailing blank line.
Mark Wooding [Fri, 28 Jul 2017 23:50:05 +0000 (00:50 +0100)]
selpk.pyx: Fix bad cut-and-paste in the callback property.
Should have been `packetproc', but was still `lineproc' presumably from
cut-and-paste from `selbuf'.
Mark Wooding [Fri, 28 Jul 2017 23:47:11 +0000 (00:47 +0100)]
ident.pyx: Remove traces of bogus `failed' and `eof' callbacks.
The `eof' callback was just a loose property and didn't actually work at
all. The `failed' callback had all of the necessary machinery but was
never invoked.
Mark Wooding [Thu, 27 Jul 2017 09:50:39 +0000 (10:50 +0100)]
codec.pyx.in: Cast arguments to `xfree'.
I malloced them, so I have to free them; but the underlying type is
`const char *'. Add the necessary casts. This is actually a legitimate
use, rather than covering a Pyrex deficiency.
These are the last compiler warnings.
Mark Wooding [Thu, 27 Jul 2017 09:49:59 +0000 (10:49 +0100)]
ident.pyx, defs.pxi: Use Pyrex `typecheck' for type checking.
Throw away the previous kludgy hacking with `PyObject_TypeCheck'.
Mark Wooding [Thu, 27 Jul 2017 09:49:10 +0000 (10:49 +0100)]
assoc.pyx, sym.pyx: Mark arguments as `not None'.
They should have been marked before, but I was careless. Apparently
this will be the default in future, which seems like a good thing.
Mark Wooding [Thu, 27 Jul 2017 09:46:14 +0000 (10:46 +0100)]
*.pyx, defs.pxi, grim.h: Add awful casts to `PyObject_AsReadBuffer'.
Muffles compiler warnings. The second argument is meant to be `const
void **' which is hard to manufacture with Pyrex. Smuggle a `const
void *' into Pyrex's brain from `grim.h' and cast the arguments.
Mark Wooding [Thu, 27 Jul 2017 09:41:22 +0000 (10:41 +0100)]
*.pyx, defs.pxi, *.c: Fixes for 64-bit builds.
Pyrex (at least nowadays) declares `PY_SSIZE_T_CLEAN', so make this be
likely to work. Also, use `socklen_t' as necessary.
Mark Wooding [Thu, 27 Jul 2017 09:37:26 +0000 (10:37 +0100)]
array.c: Add fake initializations in `pop' and `shift' to muffle warnings.
Mark Wooding [Thu, 11 May 2017 09:42:15 +0000 (10:42 +0100)]
pubkey.c: Add support for Ed448 signatures, following RFC8032.
Mark Wooding [Thu, 11 May 2017 09:42:15 +0000 (10:42 +0100)]
pubkey.c: Support the `ed2559ctx' signature scheme from RFC8032.
Main difference is the addition of a personalization string.
In the wrapper classes, forward unknown keyword arguments on to the
underlying implementation.
Mark Wooding [Thu, 11 May 2017 09:42:15 +0000 (10:42 +0100)]
Add support for SHA3 and related algorithms.
This comes in three tranches.
* There are the basic generic-interface algorithms for SHA3-*, SHAKE*,
KMAC*, etc., which basically just turn up by themselves, and the
RNGs based on SHAKE and KMAC which took a little more work.
* There's a full implementation of the cSHAKE128 and cSHAKE256 XOFs as
a new kind of object.
* Based on this, there's a full KMAC implementation, with the fiddly
bits in Python (but all the heavy lifting is done in C), with
variable-length tag and everything. Other constructions, e.g.,
TupleHash, can easily be made in the same way.
Annoyingly, KMAC can't just be made from SHAKE by multiple inheritance
because Python gets confused about how it's supposed to construct the
objects, and, in particular, which `__new__' methods are OK to use. It
seems that the relevant code is trying to use the `HEAPTYPE' flag as a
proxy for whether a type is implemented in C, which doesn't work for our
classes. So there's a bunch of ugly delegation to do.
Mark Wooding [Thu, 11 May 2017 09:42:15 +0000 (10:42 +0100)]
algorithms.c: Add basic support for Keccak[1600, n].
This takes the form of a simple object which encapsulates the
Keccak[1600, n] state and allows mix and extract operations (which
correspond to the I/O portions of absorb/squeeze and duplexing) and
step, which actually invokes the permutation to advance the state.
None of this keeps track of rate or capacity limits beyond the obvious
memory-safety checks, so you can really screw yourself if you're not
careful.
Mark Wooding [Thu, 11 May 2017 09:42:15 +0000 (10:42 +0100)]
utils.c: Raise exceptions from `convTHING' with null arguments.
This can happen as a result of using `convTHING' in an attribute `set'
function, and the Python program trying to `del' the attribute.
Unfortunately, these conversion functions are already being used in this
context, and it leads to segfaults, e.g., from
del C.Key(C.KeyFile('', C.KOPEN_WRITE | C.KOPEN_NOFILE), 0, 'k').exptime
Easy fix.
Mark Wooding [Sun, 14 May 2017 03:28:02 +0000 (04:28 +0100)]
Merge branch '1.1.x'
* 1.1.x:
Release 1.1.2.
catacomb/__init__.py: Fix up cipher etc. names better.
algorithms.c: Support the new 16-bit key-size descriptors.
group.c: Track Catacomb group internals change.
utils.c: Raise exceptions from `convTHING' with null arguments.
Return `long' objects when `int' is requested but the value won't fit.
bytestring.c: Check for cached hash more carefully.
rand.c: Careful range checking on `block' and `mp'.
*.c: Fix docstrings for methods.
Further fixing to use `Py_ssize_t' in place of int.
Conflicts:
debian/control (already wanted later catacomb-dev)
group.c (no need for compatibility with older Catacombs)
Mark Wooding [Thu, 11 May 2017 09:42:15 +0000 (10:42 +0100)]
utils.c: Raise exceptions from `convTHING' with null arguments.
This can happen as a result of using `convTHING' in an attribute `set'
function, and the Python program trying to `del' the attribute.
Unfortunately, these conversion functions are already being used in this
context, and it leads to segfaults, e.g., from
del C.Key(C.KeyFile('', C.KOPEN_WRITE | C.KOPEN_NOFILE), 0, 'k').exptime
Easy fix.
Mark Wooding [Wed, 3 May 2017 11:48:27 +0000 (12:48 +0100)]
Return `long' objects when `int' is requested but the value won't fit.
Mostly, Python handles the error from the `int' conversion and falls
back to long, but there's something weird in iteration, where if you say
for i in ...:
print '%d' % x
then the loop finishes and /then/ you get an exception for the overflow
from the failed conversion of x to an `int'.
Follow Python's actual behaviour: have `mp_tolong_checked' take an extra
argument indicating whether to throw an exception, and modify most of
the call sites to fall back to a conversion based on `mp_topylong'.
Mark Wooding [Sun, 11 Sep 2016 23:14:16 +0000 (00:14 +0100)]
Further fixing to use `Py_ssize_t' in place of int.
This addresses the remaining compiler warnings when building for 64-bit
targets.
Mark Wooding [Wed, 26 Apr 2017 10:53:05 +0000 (11:53 +0100)]
pubkey.c, catacomb/__init__.py: Add bindings for Hamburg's X448.
Mark Wooding [Thu, 26 May 2016 08:26:09 +0000 (09:26 +0100)]
pubkey.c, ...: Support Bernstein's `
Ed25519' signature scheme.
Mark Wooding [Wed, 12 Apr 2017 17:39:01 +0000 (18:39 +0100)]
pubkey.c, ...: Support Bernstein's `X25519' key-agreement algorithm.
Mark Wooding [Wed, 3 May 2017 11:48:27 +0000 (12:48 +0100)]
Return `long' objects when `int' is requested but the value won't fit.
Mostly, Python handles the error from the `int' conversion and falls
back to long, but there's something weird in iteration, where if you say
for i in ...:
print '%d' % x
then the loop finishes and /then/ you get an exception for the overflow
from the failed conversion of x to an `int'.
Follow Python's actual behaviour: have `mp_tolong_checked' take an extra
argument indicating whether to throw an exception, and modify most of
the call sites to fall back to a conversion based on `mp_topylong'.
Mark Wooding [Thu, 26 May 2016 08:26:09 +0000 (09:26 +0100)]
bytestring.c, catacomb/__init__.py: Compare for equality in constant time.
There's an explicit `ctstreq' function which just does what you wanted.
Also, `ByteString' objects now have a rich-compare method which always
compares for equality in constant time. Ordering comparisons are
variable time still.
There's a little chicanery to retain the hash function from `str'.
Also add a simple `check' method to `GHash' and `Poly1305Hash' which
compares a hsah or MAC tag in constant time and returns a boolean
result.
Mark Wooding [Thu, 26 May 2016 08:26:09 +0000 (09:26 +0100)]
*.c: Declare `PY_SSIZE_T_CLEAN'.
Now we can process large strings on 64-bit targets. Err, win?
Mark Wooding [Thu, 26 May 2016 08:26:09 +0000 (09:26 +0100)]
algorithms.c: Add support for Poly1305.
Mark Wooding [Thu, 26 May 2016 08:26:09 +0000 (09:26 +0100)]
catacomb-python.h, util.c: Support for declaring types with weird metatypes.
Mark Wooding [Mon, 12 Sep 2016 21:27:23 +0000 (22:27 +0100)]
util.c: Add conversions between Python objects and `kludge64'.
Mark Wooding [Mon, 12 Sep 2016 21:26:53 +0000 (22:26 +0100)]
catacomb-python.h: Don't inhibit 64-bit type detection any more.
We'll be using `kludge64', and the definition must properly match up
with the library.