regress: Move case-*.in opening to shlib playback_prepare

make1test needs it too

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: implement poll

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Improve sync lost msg

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Direct traceprint to stderr

We open our own FILE* because the test program might fclose actual
stderr before returning from main.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Check traceprint in Q_vb

This may be called from Hexit, as well as our own Hfoos

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Add Tensuresetup into Hexit; remove it from Q_vb in hfuzzraw

Everyone else's generated H* functions call Tensuresetup before they
call Q_vb.  So Hexit should too.

Q_vb in hfuzzraw no longer needs it then.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: playback: check poll sanity

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzz: change check1fuzz to send stderr to /dev/null

And also be less verbose when the test fails.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Rename traceprint functionality

Replace uses of stdout with uses of traceout, currently a #define.
Change some function names.

Only functional change is the change to the env var name.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: playback: Fold other Tensure calls into Tensuresetup

Rename Tensurereportfile and Tensurefuzzrawfile to be sure we didn't
miss any diredct call sites.

No significant functional change; we just do all of the idempotent
setup at each point.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Call Tensuresetup in Q_vb, just in case

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Rename Tensurerecordfile to Tensuresetup

This is much less confusing and reflects its real role better.  We
will deal with the remaining bits of Tensure<something> that aren't
Tensurerecordfile, in a moment.

 perl -i~ -pe 's/Tensurerecordfile/Tensuresetup/g' *.m4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Move up Tensurerecordfile

Pure code motion

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: makefile and shell script fixes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Allow any positive errno value

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Use syscall_sync in Hclose

This has to mirror the non-special close in hm_stdsyscall_close

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Break out syscall_sync

No functional change

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: fixes and improvements to Paddr and Pbytes

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: work on fixing Paddr

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Fix select Pfdset

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Improve sync strings

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Improve debug stdout from fuzzraw

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Introduce FUZZRAW_SYNC

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Constify FR_write

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Fix fcntl

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Fix gettimeofday

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Fix P_succfail

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Fixes to fdsets

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: hfuzzraw: Introduce P_read_dump

This really helps debugging.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: hfuzzraw: Break out Tflushstdout

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Handle select and poll return value specially

Don't read or write it to the fuzzraw stream.  Instead, in hfuzzraw,
calculate it from the returned fd set (and check it's consistent with
the timeout).

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: *.m4: Add an hm_create_nothing to hrecord.c.m4

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: *.m4: Provide for hm_rv_select and hm_rv_poll

They default to _any so there is no functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: hmacros.i4: Provide hm_rv_* in hm_create_nothing

This is convenient, and it is going to become more so.  No functional
change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: hfuzzraw: Fix close return value

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: hfuzzraw: Break out P_succfail()

Hclose needs it.  No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: playback fuzzraw generator: Get errno and result generation right

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: playback fuzzraw generator: Fix a wrong pointer passed to fwrite

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: Provide for ADNS_TEST_FUZZRAW_STDOUT_ENABLE

This generates sort-of-.sys-file output, which is useful for
debugging.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: hfuzzraw: Fix a wrong pointer passed to fread

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: hfuzzraw: Fix that Tensurerecordfile was not idempotent

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: make1fuzz: Make fuzzraw-* too

And clean and .gitignore them.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: hfuzz: Do not redirect stdout

(And anyway the dup2 was the wrong way round.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Reorganise r1test and make1fuzz some more

We are going to want make1fuzz to make the raw fuzzer input too.
That's done by the playback programs, so we need to call them.

This means it needs to make a copy of the syscall stream made by
playback_prepare.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: playback fuzzraw dump: Initial cut of raw file generator

This has not been executed so it may well not work.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: playback fuzzraw dump: Move Tshutdown

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: fuzzraw: read new fds from stream

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Make arrangements for fuzzraw of gettimeofday

Provide a hook to call in Hgettimeofday which is a stub in all but
fuzzraw.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Make special arrangements for close()

fuzzraw wants to track fds, so we can't just pass to through there.

All the other sites get a macro hm_stdsyscall_close to just get
the simple version.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: hfuzzraw work in progress

It compiles. But:
 * close() needs to be handled specially since we are maintaining
   a synthetic fd table
 * it does not work at all yet and probably has other parts missing
 * want to (optionally) generate raw feed as part of playback
 * new fds need to get the chosen fd from the stream (and check
   it for sanity)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Fix a message about length return values

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Make fcntl extra arg be int, not long

Fixes test suite on platforms where long is not same size as int.
(Although it may work on some LE platforms anyway.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Provide Tnerrnos

No users yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: hplayback: Do not tolerate hm_rv_any values out of range

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Do not crash if fd set fd out of range

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Do not provide our own malloc in fuzz mode

We expect our fuzzer to have a stunt malloc.  So just provide stubs.

This involves moving our malloc wrapper to nonfuzz.c.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Rename Tallocshutdown to Tcommonshutdown

No functional change just yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: playback: Do not accept numeric errno values except 1..255

In particular do not accept zero, which the operating system won't
produce.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Provide fuzzcat.d

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Provide Makefile targets to generate fuzzer inputs

These will be useful for seeding AFL and may be useful for other
purposes.  This also lets us check that at least with our existing
test cases, the corresponding fuzzer input files work.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
This reverts commit 44843c19f7b3ebf59aa4cda4cef9be0f5d973126.

regress: Fix skipped tests ($$ reference in Makefile)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Introduce %_fuzz executables

This takes a single input file containing command line arguments,
stdin, and syscall stream, and runs the playback on it.

This will be useful for fuzzers.

Currently nothing calls this.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Move Texit into hnonfuzz

We are going to want to do this separately in the fuzz playback.  This
avoids difficulties with adnstest (which calls Texit rather than
Hexit) and programs which return from main(), both of which we want to
capture the exit status of in the fuzz playbacks.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Allow harness code to wrap main if it wants to

The fuzzer is going to want to wrap main().  This is a bit awkward
because everyone defines it their own way (in ways which are obviously
equivalent but which trigger compiler warnings).  Our warnings mean we
have to make sure the comiler sees a declaration.

Luckily the only occurence of main() anywhere in one of the client
programs is (necessarily) the definition of main, which is always
defined to return int.  So we can arrange the expansion to produce
both declaration and definition of Hmain.

Then in hnonfuzz.c we need to declare Hmain.  (Which we can't declare
in the global header file.)  We hope that type errors in main are
unlikely.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Wrap up fopen() too

The fuzzer is going to want to wrap fopen.  So add it to the special
syscall list, and implement it with the obvious wrapper in hnonfuzz.c.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Introduce hnonfuzz.c and Ttestinputfd

We are going to want to build a fuzzer target binary which will be
very like a playback binary, but provides a bit more wrapping.

For now we introduce hnonfuzz.o and link it into all the
currently-generated executables.

To make it not be an empty translation unit, combine this with:

Break out Ttestinputfd from hplayback.c.m4.

Neither of these changes produce any functional change.  (Ttestinputfd
ends up linked into *_record as well as *_playback but this is of no
consequence.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: shlib playback_prepare: Use test -e rather than test -f

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Break out some functions from r1test into a new file

We are going to want to reuse some of this logic for parsing case-*.sys
files.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Break a line in the Makefile.in

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: Break out Tallocshutdown

No functional change

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: m1test: Use test -e rather than test -f

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

regress: r1test: Use test -e rather than test -f

This avoids some silly error messages or other mistakes.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

timeout robustness: Track start time and duration

This allows us to detect if the clock rewinds, and restart our
timeouts from the new time.  Otherwise we might try to wait a very
long time.

The result is that clock instability may now produce spurious failures
of some queries, but it should no longer cause queries to be
indefinitely delayed.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

timeout robustness: Introduce adns__timeout_{set,clear}

This abstracts away the open-coded handling of qu->timeout.
Rename the field to catch all call sites.

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

adnshost: Support --reverse in -f mode input stream

Previously this would spuriously fail an assertion.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Remove all m4 output files from the distributed source tree.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

clean targets: Delete $(TARGETS) too!

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Document 1.5.2 changes and set version number

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Update copyright dates everywhere

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

internal.h: Do not include spurious `data' symbol (!)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

If server sends answer with TC set via TCP, bail

We shouldn't use the answer; it's corrupted.  But we don't have a
recovery strategy either.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

adns_finish: Cancel parent queries first

Here, we should not cancel a query with a parent, because that leaves
the parent in a silly state (childw, but no children) which
adns__consistency complains about.

Instead, search upwards for a parent to cancel.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Consistency checks: Distinguish "entry" from exit

Many externally-facing functions are called by adns itself.  In such a
case, on entry, there may be intdone queries in flight.  This is fine.

So distinguish cc_enter from cc_exit, and check intdone only on
cc_exit.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Fix binary compatibility of string conversion of RRs with internl addr tables

Queries without adns__qtf_bigaddr use a smaller stride than the
size of our own adns_rr_addr.  This is dealt with by explicit
calculation of the addr struct using the stride from gsz_addr.

However, this circumlocution was omitted in cs_hostaddr, with the
result that adns_rr_info would go wrong for old clients.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

types.c: Pass real adns_rrtype to all cs_* functions and some csp_*

No functional change yet.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

consistency: Call adns__intdone_process in adns_afterpoll

This avoids crashes if consistency checking is turned on.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Control flow: Introduce adns__intdone_process

No functional change.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

DNS packet parsing: Slight fix when packet is truncated

If the packet is truncated, adns__findrr_anychk returns adns_s_ok,
setting *type_r to -1.  It does not guarantee to set the other
outputs.

So, in pap_findaddrs, check for this first, rather than perhaps
reading the uninitialised `ownermatched' value.

And in adns__procdgram check the type before checking the (technically
in this case undefined) class.

In practice there is no bug in actual compiled code, because in both
call sites another test will DTRT.  I don't think contemporary
compilers spot and exploit this bug for "optimisation".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

consistency check: Tolerate done children on intdone

This can happen temporarily (for example, during globalsystemfailure
or during adns__query_fail's recursion through queries).

There is no problem unless it persists as we leave adns.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

Control flow: Document restrictions on globalsystemfailure

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

internal.h: Use `unsigned' for nextid

This is constantly incremented and needs to wrap.  In practice I don't
believe any compilers spot the UB.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

internal.h: Add comment discouraging excessive MAXSERVERS

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

adns_processwriteable: assert that write did not write more than we asked

This is better than proceeding to make tcpsend.used negative.

Should this ever happens, which it doesn't except under AFL, because
of a bug in hfuzzraw.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

adns: Correct a parsing error message very slightly

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

globalsystemfailure: Do not tangle our lists

If a globalsystemfailure happens, we must remove each query from its
corresponding list, since adns__query_fail will try to remove it and
then put it on some other list.

Previously, any globalsystemfailure was likely to lead to an assertion
failure.  This doesn't seem to me to be a security problem.

Found by AFL.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

setup.c: Do not make wild pointer access if resolv.conf prefix length insane

(Found by AFL.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

setup.c: Do not fail assertion if `search' with no options in resolv.conf

(Found by AFL.)

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

src/query.c: Correct assertion in free_interim

!something is always >=0.  We meant to check that the free_interim
does not free more than was allocated.

Also do the assertion before the manipulation.  That reduces the
probability that a compiler will "prove" that the assertion is not
needed, or that it will fire too late.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

common.make.in: add -Wno-unused-value

Our GET_* macros return the value they've assigned, for convenience,
but this upsets new versions of gcc:

  warning: right-hand operand of comma expression has no effect [-Wunused-value]

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>

SECURITY: Ignore apparent answers before first RR we found the first time

This way the second answer scan finds the same RRs at the first.
Otherwise, adns can be confused by interleaving answers for the CNAME
target, with the CNAME itself.

In that case the answer data structure (on the heap) can be overrun.

With this change, we prefer to look only at the answer RRs which come
after the CNAME, which is at least arguably correct.

Found by AFL 2.35b.  CVE-2017-9109.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>