chiark / gitweb /
Ian Jackson [Mon, 17 May 2021 17:07:59 +0000 (18:07 +0100)]
apitest: Provide @examples@
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 17:05:13 +0000 (18:05 +0100)]
apitest: Prepare for at-bundles
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 16:59:01 +0000 (17:59 +0100)]
apitest: Move ok printing to end of main (portmanteau) main
This means it happens for wdt too - fine.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 16:54:55 +0000 (17:54 +0100)]
apitest: Un-clone-and-hack api early arg test predicate
At some point I guess we could make this concrete entirely but
this will do for now.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 16:52:29 +0000 (17:52 +0100)]
apitest: Make it also a portmanteau
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 16:44:51 +0000 (17:44 +0100)]
Makefile: Fix example test bundle building not to produce noise
zip simply ignores missing files, so the dupes were harmless.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 16:33:42 +0000 (17:33 +0100)]
apitest: Move portmanteau machinery from wdriver
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 16:27:25 +0000 (17:27 +0100)]
Makefile: tidy bundle output
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 16:24:19 +0000 (17:24 +0100)]
big-bundle example: Restore renaming of wikipedia
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 16:21:13 +0000 (17:21 +0100)]
big-bundle example: Comment out the scraper bits in tomls
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 16:13:35 +0000 (17:13 +0100)]
big-bundle example: Use only the bits we want
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 16:05:23 +0000 (17:05 +0100)]
media-scraper: Provide LIBRARY_FILE_INPUTS
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 15:56:35 +0000 (16:56 +0100)]
progress: Flush so the progress info is actually timely
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 15:49:01 +0000 (16:49 +0100)]
Makefile: Provide an example big bundle
We mangle the wikimedia files, ad-hoc.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 15:32:15 +0000 (16:32 +0100)]
Makefile: Clean files *.tmp
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 15:31:35 +0000 (16:31 +0100)]
Makefile: Clean the test bundle zipfile
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 14:57:36 +0000 (15:57 +0100)]
if_let: Remove the "match" from the general case
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 14:55:50 +0000 (15:55 +0100)]
bundles: Print the bundle name after we have uploaded it
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 14:55:01 +0000 (15:55 +0100)]
if_let: Provide the logically-necessary fourth case
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 10:21:31 +0000 (11:21 +0100)]
progress: Minor change to byte count output formatting
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 10:41:03 +0000 (11:41 +0100)]
progress: Use threads to do concurrent progress reports.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 13:22:25 +0000 (14:22 +0100)]
progress: Make ProgressMode come from MgmtChannel
That's where the implementation (and any restrictions) come from.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 10:36:37 +0000 (11:36 +0100)]
progress: Introduce ProgressMode
This allows progress reports to be disabled entirely.
But overall, no functional change.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 10:16:35 +0000 (11:16 +0100)]
Revert "progress: Disable the upload progress for now"
This is now done in bundles.rs
This reverts commit
07e26362b74a836d84d8c988bc0169cc3c00d749.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 10:16:17 +0000 (11:16 +0100)]
bundles progress: Rework to do the duplex disablement here
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 13:08:05 +0000 (14:08 +0100)]
mgmtchannel: Add threadsafety bounds to the bulk writer
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 13:08:13 +0000 (14:08 +0100)]
packetframe: Use concrete types in two places
This will let us inherit Send and Sync from the inner writer.
Acked-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 11:00:01 +0000 (12:00 +0100)]
progress: Call termprogress::clear() before examining cmd response
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 10:59:26 +0000 (11:59 +0100)]
progress: Provide termprogress::clear()
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 10:54:28 +0000 (11:54 +0100)]
progress: Make cmd_withbulk take termprogress::Reporter, not a Fn
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 21:55:15 +0000 (22:55 +0100)]
bundles: Truncate bundle files before deleting them
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 22:24:52 +0000 (23:24 +0100)]
if_let: Provide a way to get at the error
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 22:05:48 +0000 (23:05 +0100)]
bundles: Break out scan_game_bundles
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 21:49:22 +0000 (22:49 +0100)]
bundles: A todo
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 19:44:32 +0000 (20:44 +0100)]
cmdlistener: Apply an idle timeout and an upload timeout
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 19:44:04 +0000 (20:44 +0100)]
cmdlistener: Use a concrete type for the read stream
The write stream is still Box<dyn...>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 19:21:09 +0000 (20:21 +0100)]
Reader access: Add some methods to get at the inner TimedFdReader
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 10:02:42 +0000 (11:02 +0100)]
timedread: Remove some unnecessary braces
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 13:55:31 +0000 (14:55 +0100)]
mgmtchannel: Change contained type to a TimedFdReader
This is in preparation for timeouts.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 13:48:35 +0000 (14:48 +0100)]
timedfd: Add note re nonblock open-file
This is why we must swap both ends of a mgmtchannel at once.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 13:47:17 +0000 (14:47 +0100)]
timedfd: Provide TimedFdWriter too
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 13:47:00 +0000 (14:47 +0100)]
timedfd: Refactor to prep for writing too (2)
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 13:38:37 +0000 (14:38 +0100)]
timedfd: Refactor to prep for writing too (1)
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 13:28:16 +0000 (14:28 +0100)]
timedfd: Rename from timedread
We are going to need write too.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 13:55:19 +0000 (14:55 +0100)]
timedread: Add to prelude
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 10:01:00 +0000 (11:01 +0100)]
timedread: Retry on EINTR from poll
In test failures which occurred before I introeduced
packetframe: Fuse: Do not persist EINTR or EWOULDBLOCK
I think I observed poll() returning EINTR. Better to handle
that here even if the next layer up might do so.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Mon, 17 May 2021 09:45:08 +0000 (10:45 +0100)]
packetframe: Fuse: Do not persist EINTR or EWOULDBLOCK
Fusing EINTR is particularly toxic. BufReader retries.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 19:20:58 +0000 (20:20 +0100)]
timereader: Minor refactoring
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 19:12:53 +0000 (20:12 +0100)]
timereader: Provide useful methods
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 23:27:12 +0000 (00:27 +0100)]
timereader: mio: Add another feature which we need
Cargo feature unification seemed to have been getting this one for us
previously, anyway. But lacking it is a bit of a hazard.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 18:52:16 +0000 (19:52 +0100)]
timereader: New module - does not work properly yet
At first I tried doing this by a contraption which would dup2
/dev/null over the socket, but it turns out that (with Linux on my
laptop at least) this doesn't interrupt recvfrom in another thread.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 20:41:19 +0000 (21:41 +0100)]
cmdlistener: Break out write_error
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 17:00:58 +0000 (18:00 +0100)]
packetframe: Provide get_even_broken method
We will use this to implement AsRawFd
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 12:18:20 +0000 (13:18 +0100)]
packetframe: Fuse: retain the inner RW when broken
We are going to want this for AsRawFd, which needs to work even when
broken since it is supposed to be infallible.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 12:44:20 +0000 (13:44 +0100)]
mgmtchannel: Drop BufReader and BufWriter
FrameReader and FrameWriter have internal buffers
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sun, 16 May 2021 09:35:28 +0000 (10:35 +0100)]
progress: Disable the upload progress for now
I have realised this can deadlock. Fixing it seems to involve maybe
putting Tokio into otter(1) ...
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 23:44:10 +0000 (00:44 +0100)]
Drop some done todos
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 23:36:04 +0000 (00:36 +0100)]
Drop some todos: Library listing
This has been sorted out.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 23:35:01 +0000 (00:35 +0100)]
Drop a todo: MultiSpec
We don't want to dedupe this; the list of items is completely fixed by
the input.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 23:32:38 +0000 (00:32 +0100)]
errors: Abolish obsolete MgmtError::GameCorrupted
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 22:53:01 +0000 (23:53 +0100)]
otter: Dedup library items before adding them
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 23:10:37 +0000 (00:10 +0100)]
bundles: No, we don't need to delete them, they'll go later
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 23:10:01 +0000 (00:10 +0100)]
bundles: Punt on src field in metadata
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 23:09:02 +0000 (00:09 +0100)]
otter: Show bundle in library information
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 23:01:25 +0000 (00:01 +0100)]
otter: Uwe the prelude alias for ItemSpec
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 23:00:50 +0000 (00:00 +0100)]
shapelib: Provide and use ItemEnquiryData -> ItemSpec
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 22:58:51 +0000 (23:58 +0100)]
shapelib: Put LibraryEnquiryData in ItemEnquiryData
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 22:55:19 +0000 (23:55 +0100)]
shapelib: Put *EnquiryData in the prelude
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 22:42:57 +0000 (23:42 +0100)]
library listing: Rework library patterns
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 22:42:52 +0000 (23:42 +0100)]
otter: Change to fehler in a function
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 22:39:47 +0000 (23:39 +0100)]
library listing: Provide for listing libs on the server side
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 21:21:37 +0000 (22:21 +0100)]
ItemEnquiryData: Put the lib name in it
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 20:57:44 +0000 (21:57 +0100)]
ItemEnquiryData: format it with Display, not an ad-hoc method
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 18:41:38 +0000 (19:41 +0100)]
CHANGELOG: wip
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 20:35:13 +0000 (21:35 +0100)]
shapelib; Further improve error printing
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 20:08:14 +0000 (21:08 +0100)]
command: Further improve error printing
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:57:16 +0000 (20:57 +0100)]
gamestate: Further improve error printing
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:57:10 +0000 (20:57 +0100)]
otter: Further improve error printing
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:52:29 +0000 (20:52 +0100)]
bundles: Further improve error printing
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:51:06 +0000 (20:51 +0100)]
spec: Further improve error printing
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:45:40 +0000 (20:45 +0100)]
otter: Further improve error printing
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:40:17 +0000 (20:40 +0100)]
otter: Further improve error printing
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:33:23 +0000 (20:33 +0100)]
otter: Do not panic on (most) command execution errors
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:26:44 +0000 (20:26 +0100)]
shapelib: Check outline is valid
Otherwise we fail later with an internal error and dump a stack trace
to the client.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:17:57 +0000 (20:17 +0100)]
example bundle: Tune the lemon
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:08:02 +0000 (20:08 +0100)]
shapelib: Note a bug!
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 19:01:43 +0000 (20:01 +0100)]
example bundle: Make the lemon a sensible size
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 18:20:29 +0000 (19:20 +0100)]
bundles: Test shape loading from bundle
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 16:35:31 +0000 (17:35 +0100)]
bundles: Actually implement shape loading
A small amount of glue!
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 18:39:57 +0000 (19:39 +0100)]
shapelib: Break out Registry::add()
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 18:11:22 +0000 (19:11 +0100)]
shapelib: Provide per-game registry iterator
Currently this is juut the global registry.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 18:30:12 +0000 (19:30 +0100)]
apitest: Break out some_library_add
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 18:24:45 +0000 (19:24 +0100)]
apitest: Break out place_library_load_markers
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 18:06:15 +0000 (19:06 +0100)]
Make library list a per-table thing at managemnet API
The LibraryListByGlob command now takes a game name, and
acquires an &InstanceGuard. Currently, it ignores it.
The list_items method is now on MgmtChannelForGame.
otter(1) LibGlobArgs now has the table name. The library-list
subcommand accesses the game (and has different arguments)
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 18:04:40 +0000 (19:04 +0100)]
apitest: Test library listing
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 17:45:24 +0000 (18:45 +0100)]
cmdlistener: Use access_bundles in two more places
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 17:43:38 +0000 (18:43 +0100)]
cmdlistener: Rename access_bundles
This is not just for modifying.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 17:37:32 +0000 (18:37 +0100)]
cmdlistener: Rename start_access_game
This is not just for modifying.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 16:30:24 +0000 (17:30 +0100)]
lib enquiries: Some todos
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Ian Jackson [Sat, 15 May 2021 16:20:09 +0000 (17:20 +0100)]
bundles: Reset the access key on clear
This invalidates the old urls for the old bundles.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
~ianmdlvl / otter.git / log