From: Ian Jackson Date: Sat, 4 Jul 2009 17:29:08 +0000 (+0100) Subject: Split out README.files; docs improvements; tidy up a leftover file X-Git-Tag: 1.9.2~32 X-Git-Url: http://www.chiark.greenend.org.uk/ucgi/~yarrgweb/git?p=ypp-sc-tools.web-live.git;a=commitdiff_plain;h=fc71bd8c21ff11ded6888512ec30a8506d7ed1ec Split out README.files; docs improvements; tidy up a leftover file --- diff --git a/pctb/README b/pctb/README index 2a085b2..185a022 100644 --- a/pctb/README +++ b/pctb/README @@ -7,15 +7,17 @@ This tool can: - upload the results to PCTB To run it, change to this directory, type `make', and then: - ./ypp-commodities --tsv >commods.tsv -or ./ypp-commodities --upload +to upload to the PCTB server. Currently we use pctb.ilk.org, +the testing instance of the PCTB database, pending approval from the +operators of the main server. + +Or, for example, for a tab-separated values dump: + ./ypp-commodities --tsv >commods.tsv While it is capturing the screenshots, do not move the mouse or use the keyboard. Keyboard focus must stay in the YPP client window. -You will probably need to turn off `Use antialiased font' in the YPP -client. This is in the Ye panel, Options, tab `General'. Command-line options -------------------- @@ -59,96 +61,6 @@ Options to override which servers we talk to: Or set the environment variables YPPSC_PCTB{_PCTB, _DICT_UPDATE, _DICT_SUBMIT} -Files we use and update ------------------------ - -The program reads and writes the following files: - - * #pages#.pnm - - Contains one or more images (as raw ppms, end-to-end) which are the - screenshots taken in the last run. This is (over)written whenever - we take screenshots from the YPP client. You can reprocess an - existing set of screenshots with the --same (aka --analyse-only) - option; in that case we just read the screenshots file. - - You can specify a different file with --screenshot-file. - - If you want to display the contents of this file, `display' can do - it. Don't try `display vid:#pages#.pnm' as this will consume - truly stupendous quantities of RAM - it wedged my laptop. - - * #master-newcommods#.txt #local-newcommods#.txt - - Dictionary of newly introduced commodities. When a new commodity - appears in Puzzle Pirates, the PCTB server operators need to add it - to their database for us to be able to upload data about it. - - It can sometimes take a few days to do this. In the meantime, it - is possible to upload partial data - data just omitting that - commodity. This is controlled by these files: they list - commodities which should be automatically ignored if the PCTB - server doesn't know about them. The master file is downloaded and - updated automatically from my server. You may create the local - file yourself. The format is simple: one commodity per line. - - Unrecognised commodities can also be due to OCR failure so - double-check what you're doing before overriding the uploader by - telling it to ignore an unrecognised commodity. - - * #master-reject#.txt #local-reject#.txt - - Dictionary of regexps which, when the OCR appears to match, we - reject instead. At the moment this is used to stop us thinking - that `Butterfly weed' is `Butterflyweed'. This happens if the - character set dictionary is missing the lowercase `y ' glyph. - See README.charset. - - * #master-char*#.txt #local-char*#.txt - #master-pixmap#.txt #local-pixmap#.txt - - Character set and image dictionaries. For the semantics of the - char* files README.charset. There is not currently any accurate - documentation of this dictionary format. - - #master-*#.txt contain the centrally defined and approved data. - They are downloaded automatically from the SC PCTB server and - updated each run. You can safely delete this file, if everything - is online, if you want to fetch a fresh copy. - - #local-*#.txt are a local copy of your submissions, so that they - will be used by your client pending approval by me. You can delete - this file if you think you may have made a mistake. - - See README.privacy for details of the communications with the SC - server about the contents of these dictionaries. - - * #commodmap#.tsv - - Map from commodity names to the numbers required by the PCTB - server. This is fetched and updated automatically as necessary. - It can safely be deleted as it will then be refetched. - - * #upload-1#.html #upload-2#.html - - We screenscrape the pages from the PCTB upload server. The actual - HTML returned from the upload server is left in these dropping - files for debugging etc. - - * .tmp - - When any of these tools overwrite one of the persistent dictionary - files, they temporarily write to .tmp. - -These files are all in the current working directory. There is not -yet any feature to have them be somewhere else. The helper programs - dictionary-manager - commod-results-processor -must (currently) also be in the current directory. - -Future versions may have more helpers and more data files. - - Installation requirements ------------------------- @@ -170,6 +82,17 @@ This program has quite a few dependencies: On other Linux distros the packages may have different names, but these should be roughly right for Debian and its derivatives. +The supplied helper programs + dictionary-manager + commod-results-processor + database-info-fetch +must (currently) also be in the current working directory when you run +the main ypp-commodities program. + +The data files (see README.files) are also left in the current working +directory. There is notyet any feature to have the data files and +helpers be somewhere else. + Reporting problems ------------------ diff --git a/pctb/README.files b/pctb/README.files new file mode 100644 index 0000000..7622913 --- /dev/null +++ b/pctb/README.files @@ -0,0 +1,82 @@ +Files we use and update +----------------------- + +The program reads and writes the following files: + + * #pages#.pnm + + Contains one or more images (as raw ppms, end-to-end) which are the + screenshots taken in the last run. This is (over)written whenever + we take screenshots from the YPP client. You can reprocess an + existing set of screenshots with the --same (aka --analyse-only) + option; in that case we just read the screenshots file. + + You can specify a different file with --screenshot-file. + + If you want to display the contents of this file, `display' can do + it. Don't try `display vid:#pages#.pnm' as this will consume + truly stupendous quantities of RAM - it wedged my laptop. + + * #master-newcommods#.txt #local-newcommods#.txt + + Dictionary of newly introduced commodities. When a new commodity + appears in Puzzle Pirates, the PCTB server operators need to add it + to their database for us to be able to upload data about it. + + It can sometimes take a few days to do this. In the meantime, it + is possible to upload partial data - data just omitting that + commodity. This is controlled by these files: they list + commodities which should be automatically ignored if the PCTB + server doesn't know about them. The master file is downloaded and + updated automatically from my server. You may create the local + file yourself. The format is simple: one commodity per line. + + Unrecognised commodities can also be due to OCR failure so + double-check what you're doing before overriding the uploader by + telling it to ignore an unrecognised commodity. + + * #master-reject#.txt #local-reject#.txt + + Dictionary of regexps which, when the OCR appears to match, we + reject instead. At the moment this is used to stop us thinking + that `Butterfly weed' is `Butterflyweed'. This happens if the + character set dictionary is missing the lowercase `y ' glyph. + See README.charset. + + * #master-char*#.txt #local-char*#.txt + #master-pixmap#.txt #local-pixmap#.txt + + Character set and image dictionaries. For the semantics of the + char* files README.charset. There is not currently any accurate + documentation of this dictionary format. + + #master-*#.txt contain the centrally defined and approved data. + They are downloaded automatically from the SC PCTB server and + updated each run. You can safely delete this file, if everything + is online, if you want to fetch a fresh copy. + + #local-*#.txt are a local copy of your submissions, so that they + will be used by your client pending approval by me. You can delete + this file if you think you may have made a mistake. + + See README.privacy for details of the communications with the SC + server about the contents of these dictionaries. + + * #commodmap#.tsv + + Map from commodity names to the numbers required by the PCTB + server. This is fetched and updated automatically as necessary. + It can safely be deleted as it will then be refetched. + + * #upload-1#.html #upload-2#.html + + We screenscrape the pages from the PCTB upload server. The actual + HTML returned from the upload server is left in these dropping + files for debugging etc. + + * .tmp + + When any of these tools overwrite one of the persistent dictionary + files, they temporarily write to .tmp. + +Future versions may have more helpers and more data files. diff --git a/pctb/setisland-history b/pctb/setisland-history deleted file mode 100644 index 4935a94..0000000 --- a/pctb/setisland-history +++ /dev/null @@ -1,28 +0,0 @@ - 1 cd things/ypp-sc-tools-pctb/pctb/ - 2 curl 'file:///home/ian/things/ypp-sc-tools-pctb/pctb/upload.php?topisland=560&ocean=2&island=48&action=setisland&forcereload=1244747001&filename=FMa4qY' - 3 curl 'http://pctb.ilk.org/pctb/upload.php?topisland=560&ocean=2&island=48&action=setisland&forcereload=1244747001&filename=FMa4qY' >v.html - 4 less v.html - 5 curl 'http://pctb.ilk.org/upload.php?topisland=560&ocean=2&island=48&action=setisland&forcereload=1244747001&filename=FMa4qY' >v.html - 6 less v.html - 7 curl 'http://pctb.ilk.org/upload.php?topisland=560&ocean=2&island=48&action=setisland&forcereload=1244748679&filename=PFayDW' >v.html - 8 less v.html - 9 less raw.tsv - 10 less v.html - 11 git-status - 12 git-diff - 13 git-branch - 14 git-checkout -b temp.marketdata-examples - 15 git-add raw.tsv u v.html marketdata.gz marketdata-example marketdata-example2 - 16 rm commodmap.php - 17 git-add capture.dat - 18 git-status - 19 git-commit -m 'Various example files' - 20 git-commit -m 'First upload stage' yppsc-commod-processor - 21 git-status - 22 git-checkout master - 23 display \#pages.c# - 24 display \#pages#.ppm - 25 bg - 26 display \#pages-market#.ppm - 27 history |less - 28 history >setisland-history