X-Git-Url: https://www.chiark.greenend.org.uk/ucgi/~yarrgweb/git?a=blobdiff_plain;f=pctb%2FREADME;h=e6d7707616b4049b2f1d40d3e7957fc70e1bf93e;hb=2c3d2e7de4d4aea3536ea79ced23677e24589833;hp=9402240f4845ef5a7df0fde465dacc9def9597f0;hpb=6e70a6f2ac5765379239252d771f9218a80b9c71;p=ypp-sc-tools.db-test.git diff --git a/pctb/README b/pctb/README index 9402240..e6d7707 100644 --- a/pctb/README +++ b/pctb/README @@ -4,9 +4,15 @@ Overview This tool can: - screenscrape the commodities trading screen - produce the results as a tab separated values file - - **TODO** upload the results to PCTB + - upload the results to PCTB To run it, change to this directory, type `make', and then: + ./ypp-commodities --upload +to upload to the PCTB server. Currently we use pctb.ilk.org, +the testing instance of the PCTB database, pending approval from the +operators of the main server. + +Or, for example, for a tab-separated values dump: ./ypp-commodities --tsv >commods.tsv While it is capturing the screenshots, do not move the mouse or use @@ -27,64 +33,32 @@ Options to vary the processing: --quiet Suppress progress messages --screenshot-file F Store or read screenshots in F rather than #pages#.pnm --window-id ID Specified X window is the YPP client - do not search - --edit-charset Enable character set editing. See README.charset. - -Controlling what happens to the results: + --edit-charset Enable character set editing. See README.dictionary. + --no-edit-charset Do not edit charset even if #local-char*#.txt exists. + --find-island Find and print the ocean and island. Suppresses OCR + and output unless used with result processing option. + --test-servers Set default servers to be the test servers, not + the real live ones (doesn't affect explicit settings). + +Controlling what happens to the results - only one at a time: --upload (default) Upload to the PCTB server --tsv Print data as clean tab-separated-values file --raw-tsv Dump the raw (not deduped, unsorted) OCR'd data --best-prices Print best buy and sell price for each commodity --arbitrage Print arbitrage opportunities +Privacy options, which control conversations with the dictionary server: + --dict-local-only * Do not talk to the server even to fetch new dictionary. + --dict-read-only * Only fetch new dictionary, do not submit new entries. + --dict-anon Don't quote pirate name if submitting entries. + --dict-submit Submit entries quoting my pirate name. (default) +Please do not use options marked * with --upload. See README.privacy. -Files we use and update ------------------------ - -The program reads and writes the following files: - - * #pages#.pnm - - Contains one or more images (as raw ppms, end-to-end) which are the - screenshots taken in the last run. This is (over)written whenever - we take screenshots from the YPP client. You can reprocess an - existing set of screenshots with the --same (aka --analyse-only) - option; in that case we just read the screenshots file. - - You can specify a different file with --screenshot-file. - - If you want to display the contents of this file, `display' can do - it. Don't try `display vid:#pages#.pnm' as this will consume - truly stupendous quantities of RAM - it wedged my laptop. - - * charset-15.txt - - Character set database. For the semantics of the contents of this - file see README.charset. There is not currently any accurate - documentation of this database format. - - If you delete this file you'll have to re-enter a lot of glyph data - (and probably get it wrong and make the program misrecognise - things). If you want to undo any mistakes you may have made - answering OCR questions you can safely revert this to the version - I've supplied. - - * #commodmap#.tsv - - Map from commodity names to the numbers required by the PCTB - server. This is fetched and updated automatically as necessary. - It can safely be deleted as it will then be refetched. - * .new - - When any of these tools overwrite one of the persistent database - files, they temporarily write to .new. - -These files are all in the current working directory. There is not -yet any feature to have them be somewhere else. The helper programs - yppsc-ocr-resolver - yppsc-commod-processor -must (currently) also be in the current directory. - -Future versions may have more helpers and more data files. +Options to override which servers we talk to: + --pctb-server HOST|URL Talk to the PCTB server at HOST or URL. + --dict-submit-url URL Submit dictionary entries with HTTP POST under URL. + --dict-update-from SRC Fetch updated master dictionary with rsync from SRC. +Or set the environment variables YPPSC_PCTB{_PCTB, _DICT_UPDATE, _DICT_SUBMIT} Installation requirements @@ -98,6 +72,7 @@ This program has quite a few dependencies: - pnm command line utilities for image manipulation netpbm - X11 libraries, including dev files for building libx11-dev - XTEST library, including dev files for building libxtst-dev + - Perl-compatible regexp library, including dev files libpcre3-dev - Tk interpreter /usr/bin/wish tk8.4 - Perl module XML::Parser libxml-parser-perl - Perl module JSON::Parser libjson-perl @@ -105,7 +80,21 @@ This program has quite a few dependencies: - Perl interpreter and basic modules perl (usu.installed) On other Linux distros the packages may have different names, but -these should be roughly right for Debian and its derivatives. +these should be roughly right for Debian and its derivatives. You can +install them with this rune: + sudo apt-get install build-essential libnetpbm10-dev netpbm libx11-dev libxtst-dev libpcre3-dev tk8.4 libxml-parser-perl libjson-perl + + +The supplied helper programs + dictionary-manager + commod-results-processor + database-info-fetch +must (currently) also be in the current working directory when you run +the main ypp-commodities program. + +The data files (see README.files) are also left in the current working +directory. There is notyet any feature to have the data files and +helpers be somewhere else. Reporting problems @@ -127,27 +116,24 @@ error messasge. I'll then be able to understand what's wrong, hopefully. -Phoning home - privacy ----------------------- +Privacy +------- The main purpose of this program is to connect to the PCTB server and -upload data. The program does not currently phone home at all in -modes other than --upload, and when it does it connects to the -PCTB server not to a system of mine. +upload data. It will do that if you run it with --upload. -However, there are some improvements which I may introduce in the -future which may change this. I am considering: +This program will also, by default, talk to the dictionary server I +have set up: to download updated image dictionaries, and to upload new +dictionary entries which you create with the PCTB client dictionary +GUI. This feature is mentioned in and controllable in the GUI itself, +so it won't happen without you knowing about it. - * Having the ocr character resolver talk to a server run by me - to look for missing glpyhs, and/or upload those glyphs back - to that server so that they can be shared. +The uploads will by default mention your ocean and pirate name; if you +don't want that, pass the --dict-anon option, or untick the box in the +GUI. - * Having the upload client upload a copy of the data to a server run - by me, when run in --upload mode. +See README.privacy for full details. -If I do do this these new functions may be enabled by default, but it -will be possible to turn them off, or direct them to different -servers, with command-line options, and they will be documented here. - Ian Jackson