chiark / gitweb /
Hans-Christoph Steiner [Wed, 22 Jul 2015 16:59:50 +0000 (09:59 -0700)]
in JSON metadata, convert unicode to str to match the internal format
fdroidserver currently uses plain python 2.x strings rather than unicode
strings, so the parsed JSON should produce the same format as the parsed
.txt metadata.
Hans-Christoph Steiner [Wed, 22 Jul 2015 08:16:43 +0000 (01:16 -0700)]
add test for metadata parsing based on a parse output to a pickle
This is a test to cover future modifications of the .txt metadata parsing.
The pickle file was generated by just dumping the current parsed metadata,
so this test will always succeed if the parsing is not changed.
Hans-Christoph Steiner [Wed, 22 Jul 2015 08:12:21 +0000 (01:12 -0700)]
move metadata post-parse processing to reusable function
Hans-Christoph Steiner [Wed, 22 Jul 2015 06:19:56 +0000 (23:19 -0700)]
add test files to pre-commit hook and fix pep8 errors
Hans-Christoph Steiner [Tue, 21 Jul 2015 18:50:49 +0000 (11:50 -0700)]
support app metadata in JSON format
While the current text metadata format is good for human readability and
editability, it is difficult to produce and parse using code. JSON is a
widespread standard format for easy automatic parsing and creating, while
having decent human readability.
Hans-Christoph Steiner [Tue, 21 Jul 2015 18:46:01 +0000 (11:46 -0700)]
use common method to set up defaults in metadata dict
This is needed for the upcoming metadata formats: JSON, XML, etc.
Hans-Christoph Steiner [Mon, 20 Jul 2015 23:55:15 +0000 (16:55 -0700)]
metadata: rename parse_metadata() to parse_txt_metadata()
This makes the code clear as support for other metadata formats is added.
Daniel Martí [Tue, 1 Sep 2015 00:05:08 +0000 (17:05 -0700)]
Move requests code out of common.py, closes #114
Daniel Martí [Mon, 31 Aug 2015 22:39:07 +0000 (15:39 -0700)]
lint: fixed list false positives, no need for utf
Daniel Martí [Mon, 31 Aug 2015 22:32:52 +0000 (15:32 -0700)]
lint: warn about duplicate lines
Daniel Martí [Mon, 31 Aug 2015 22:25:41 +0000 (15:25 -0700)]
lint: catch more incorrect lists
Daniel Martí [Sun, 30 Aug 2015 23:29:03 +0000 (16:29 -0700)]
lint: stop accepting Office as valid category
Daniel Martí [Sun, 30 Aug 2015 23:08:22 +0000 (16:08 -0700)]
lint: accept new category Sports & Health
Daniel Martí [Sun, 30 Aug 2015 03:19:32 +0000 (20:19 -0700)]
Follow-up to
5a0129d8: also replace file mentions
Daniel Martí [Sun, 30 Aug 2015 01:37:08 +0000 (18:37 -0700)]
lint: accept all new categories
Daniel Martí [Sun, 30 Aug 2015 01:24:18 +0000 (18:24 -0700)]
lint: new category
Daniel Martí [Sun, 30 Aug 2015 00:56:45 +0000 (17:56 -0700)]
lint: Wallpaper is now called Theming
Daniel Martí [Sun, 30 Aug 2015 00:52:15 +0000 (17:52 -0700)]
lint: remove Children as valid category
Daniel Martí [Sat, 29 Aug 2015 02:20:39 +0000 (19:20 -0700)]
Move scan_source into scanner.py
Not really a common.py thing.
Daniel Martí [Sat, 29 Aug 2015 02:05:47 +0000 (19:05 -0700)]
lint: complain about unwrapped descriptions
Daniel Martí [Sat, 29 Aug 2015 01:37:23 +0000 (18:37 -0700)]
Simplify some file logic with "with"
Daniel Martí [Sat, 29 Aug 2015 01:29:28 +0000 (18:29 -0700)]
stats: better exit message
Daniel Martí [Sat, 29 Aug 2015 01:28:39 +0000 (18:28 -0700)]
stats: write known apks in natural order
By using natural instead of alphabetical sorting, we support sorting by
vercodes properly and thus app versions show in the correct order.
Daniel Martí [Sat, 29 Aug 2015 01:26:23 +0000 (18:26 -0700)]
Simlify known_apks writing
Daniel Martí [Fri, 28 Aug 2015 23:14:53 +0000 (16:14 -0700)]
GitLab CI: Java is required to set up the SDK
Daniel Martí [Fri, 28 Aug 2015 23:11:54 +0000 (16:11 -0700)]
Gitlab CI: install SDK
Daniel Martí [Fri, 28 Aug 2015 23:02:19 +0000 (16:02 -0700)]
Gitlab CI: run tests too
Daniel Martí [Fri, 28 Aug 2015 22:53:59 +0000 (15:53 -0700)]
Add missing -v and -q to readmeta
Daniel Martí [Fri, 28 Aug 2015 22:53:10 +0000 (15:53 -0700)]
lint: better footer output logic
Daniel Martí [Fri, 28 Aug 2015 22:50:34 +0000 (15:50 -0700)]
lint: warn about unlinkified links
Daniel Martí [Fri, 28 Aug 2015 22:48:53 +0000 (15:48 -0700)]
metadata: error on url titles that aren't
Daniel Martí [Fri, 28 Aug 2015 21:23:30 +0000 (14:23 -0700)]
lint: complain if link url isn't a url
Moving this check from metadata into lint is perhaps overkill, but at least it
doesn't break fdroid update because of the maintainer notes.
Daniel Martí [Fri, 28 Aug 2015 21:10:05 +0000 (14:10 -0700)]
Revert "metadata: complain if link url isn't a url"
I already tried this once. It had to be reverted because Maintainer Notes
often don't obey this. I had forgotten.
This reverts commit
97ba94903e8896ed9c3b2c8980d218141a5fbdb0.
Daniel Martí [Fri, 28 Aug 2015 18:09:37 +0000 (11:09 -0700)]
lint: format descriptions too
Like readmeta. This makes it error if any links are badly formatted or wrong.
Daniel Martí [Fri, 28 Aug 2015 18:06:46 +0000 (11:06 -0700)]
metadata: complain if link url isn't a url
Daniel Martí [Fri, 28 Aug 2015 16:10:36 +0000 (16:10 +0000)]
Merge branch 'native' into 'master'
wp-fdroid: Don't imply 32bit code does not work on 64bit systems
Maybe someone could also check if the client lists app with 32bit code as incompatible with 64bit systems, even if they can run it (arm->arm64, i386->x64, mips->mips64 etc.).
Ref: https://f-droid.org/forums/topic/accordion-and-melodeon-2/
See merge request !68
Boris Kraut [Fri, 28 Aug 2015 14:10:24 +0000 (16:10 +0200)]
wp-fdroid: Don't imply 32bit code does not work on 64bit systems
Daniel Martí [Fri, 28 Aug 2015 03:41:18 +0000 (20:41 -0700)]
metadata: fix harmless typo
Daniel Martí [Thu, 27 Aug 2015 23:08:36 +0000 (16:08 -0700)]
Add build status badge, closes #99
Daniel Martí [Thu, 27 Aug 2015 23:07:45 +0000 (16:07 -0700)]
Gitlab CI: Rename static analyzers task to "check"
Daniel Martí [Thu, 27 Aug 2015 22:59:46 +0000 (15:59 -0700)]
First try at Gitlab CI integration
Ciaran Gultnieks [Thu, 27 Aug 2015 18:04:13 +0000 (18:04 +0000)]
Merge branch 'per-app-repos' into 'master'
config option to enable per-app repos for nightly builds
For Guardian Project, we've been running an fdroid repo for the nightly builds for each of our apps: https://dev.guardianproject.info/debug This is built using a big, hacked up shell script: [update-debug-fdroid-repo](https://github.com/guardianproject/fdroid-repo-tools/blob/master/update-debug-fdroid-repo). It has proven very useful to us to be able to subscribe to the nightly build for a single app, so this the first step of porting that horrid shell script to `fdroidserver`.
This also helps make the fdroidserver tool suite the single set of tools for all types of builds and releases. That will hopefully drive more free software developers to make f-droid.org the core channel for official releases.
See merge request !66
Ciaran Gultnieks [Thu, 27 Aug 2015 10:12:22 +0000 (10:12 +0000)]
Ensure makebuildserver provisions on an update
Daniel Martí [Thu, 27 Aug 2015 01:57:13 +0000 (18:57 -0700)]
Also force https on bitbucket
Daniel Martí [Thu, 27 Aug 2015 01:35:50 +0000 (18:35 -0700)]
Don't use lint to warn about google code
There are just too many. There is a script in fdroiddata to find google code
repos that have moved to github.
Hans-Christoph Steiner [Wed, 26 Aug 2015 22:28:47 +0000 (00:28 +0200)]
standardize on HTTPS for github.com links
fdroid link already checks HTTPS in build metadata, so this should also use
HTTPS. It makes firewall rules easier when everything is HTTPS.
Ciaran Gultnieks [Wed, 26 Aug 2015 12:49:35 +0000 (12:49 +0000)]
Merge branch 'caching-for-makebuildserver' into 'master'
two quick kinds of caching for makebuildserver
I've been running `./makebuildserver` on a machine that only connects to the internet via Tor. That means slow internet and other issues. These two small changes made it easier to run `./makebuildserver` in this setup. In combination with !25, this would give good support for working in low bandwidth environments.
See merge request !67
Hans-Christoph Steiner [Wed, 26 Aug 2015 11:26:22 +0000 (13:26 +0200)]
makebuildserver: allow setting apt mirror from command line
On some setups, using a custom apt mirror is essential, so this adds a
command line flag to override the default one:
http://ftp.uk.debian.org/debian/
For example, someone who runs a local mirror for offline and low bandwidth
situations.
This uses a % rather than a .format() to avoid escaping { and }, which have
meaning when using .format().
Hans-Christoph Steiner [Wed, 26 Aug 2015 10:23:41 +0000 (12:23 +0200)]
makebuildserver: allow a list/tuple for baseboxurl to support local copy
config.vm.box_url can be a list/tuple of URLs, which is useful to specific
a locally cached copy. This is needed on slow connections, so that if it
fails, the download of jessie32.box does not have to start from the
beginning of the file again.
Hans-Christoph Steiner [Wed, 26 Aug 2015 12:44:36 +0000 (12:44 +0000)]
Merge branch 'support-vagrant-cachier' into 'master'
Add optional support for vagrant-cachier plugin
Building the basebox is excruciating for people on slow connections. I'm particularly sensitive to this after living in Central America for awhile :)
This won't affect anyone who hasn't installed the plugin. For those who do, it creates a persistent shared folder for each box (ie. testing23.box) and detects directories to cache between VM builds (apt, gems, pip, chef cache, etc.)
(The only downside is that, for those following server setup does who are not aware what vagrant-cachier does, it might be unexpected that artifacts persist between vagrant destroys.)
See merge request !25
Hans-Christoph Steiner [Tue, 25 Aug 2015 14:29:03 +0000 (16:29 +0200)]
server: switch Amazon AWS S3 upload to streaming mode
This keeps memory usage low because it only has to read a chunk at a time
into memory while before it read the whole file into memory before
uploading it.
This also seems to handle setting the permissions ACL better.
Hans-Christoph Steiner [Thu, 20 Aug 2015 15:40:18 +0000 (17:40 +0200)]
new config option: per-app repos to support nightly build repos
For devs that want to build and distribute nightly builds of their apps
using the fdroid tools. The core idea here is to make the fdroidserver
tool suite the single set of tools for all types of builds and releases.
That will hopefully drive more free software developers to make f-droid.org
an core channel for official releases.
Hans-Christoph Steiner [Tue, 11 Aug 2015 21:47:13 +0000 (23:47 +0200)]
scan_apks() sets cachechanged, so no tricks needed
Daniel Martí [Mon, 24 Aug 2015 23:34:39 +0000 (16:34 -0700)]
lint: warn about /master/ usage
Daniel Martí [Mon, 24 Aug 2015 23:24:05 +0000 (16:24 -0700)]
checkupdates: be more verbose with tags
Daniel Martí [Mon, 24 Aug 2015 22:54:05 +0000 (15:54 -0700)]
Add support for gradle properties via gradleprops
Daniel Martí [Mon, 24 Aug 2015 17:54:20 +0000 (10:54 -0700)]
Rename COPYING to LICENSE
See fdroid/fdroidclient#384
Daniel Martí [Thu, 20 Aug 2015 16:56:14 +0000 (09:56 -0700)]
Prepare for sdk level 23
Daniel Martí [Thu, 20 Aug 2015 16:42:10 +0000 (09:42 -0700)]
config: bump default build_tools after update in BS
Daniel Martí [Thu, 20 Aug 2015 16:41:47 +0000 (09:41 -0700)]
Remove default config settings from the BS
Daniel Martí [Thu, 20 Aug 2015 16:39:38 +0000 (16:39 +0000)]
Merge branch 'two-fixes' into 'master'
Two fixes
I pulled two commits out of !57 since it would be good to have them included, and specifically the `buildToolsVersion` issue was just discussed.
The current state of the `buildToolsVersion` replacement is not good. It will replace this:
```
buildToolsVersion '21.0.1'
```
with this:
```
buildToolsVersion = '21.0.1'
```
That is incorrect, since the first form is more like a function call than a variable assignment. So most likely that is causing the `buildToolsVersion` to be unset. The commit here includes a fix that works for both the rare, non-standard variable assignment way, i.e. `buildToolsVersion = '21.0.1'` as well as the standard way: `buildToolsVersion '21.0.1'`. It also includes tests for both cases.
See merge request !65
Hans-Christoph Steiner [Tue, 11 Aug 2015 13:33:03 +0000 (15:33 +0200)]
build: improve regsub pattern for setting buildToolsVersion
This addresses the discussion in !64
https://gitlab.com/fdroid/fdroidserver/merge_requests/64
Sometimes, buildToolsVersion is a kind of gradle macro call, and other
times it is a variable assignment. This regsub pattern now handles both of
those cases.
Hans-Christoph Steiner [Wed, 5 Aug 2015 22:19:30 +0000 (00:19 +0200)]
fix syntax error in manual generation and add ./gendocs.sh to the tests
Daniel Martí [Wed, 19 Aug 2015 06:46:35 +0000 (23:46 -0700)]
Only do the magic import/except dance once
No need to do it for every file, once per app is enough
Daniel Martí [Wed, 19 Aug 2015 01:13:18 +0000 (18:13 -0700)]
lint: also do http warnings on descriptions
Daniel Martí [Wed, 19 Aug 2015 01:07:01 +0000 (18:07 -0700)]
lint: better type safety
Daniel Martí [Wed, 19 Aug 2015 00:54:32 +0000 (17:54 -0700)]
Warn about url shorteners being used
Daniel Martí [Wed, 19 Aug 2015 00:53:56 +0000 (17:53 -0700)]
Warn about misplaced flattr donation links
Daniel Martí [Tue, 18 Aug 2015 23:28:58 +0000 (16:28 -0700)]
import: remove support for gitorious
Same as google code.
Daniel Martí [Tue, 18 Aug 2015 23:26:12 +0000 (16:26 -0700)]
import: remove google code support
Google code is dead and going read-only. import is just a helper program, and
we don't want to help people add apps from google code anymore. Apps should
move to other hosting sites.
Daniel Martí [Tue, 18 Aug 2015 21:01:14 +0000 (14:01 -0700)]
lint: Update Check Name with unchanged app id
Daniel Martí [Tue, 18 Aug 2015 20:40:21 +0000 (13:40 -0700)]
Add gradle 2.6 to the buildserver
Daniel Martí [Tue, 18 Aug 2015 20:39:00 +0000 (13:39 -0700)]
Update buildserver for android-23
Daniel Martí [Tue, 18 Aug 2015 05:04:20 +0000 (22:04 -0700)]
Fix type issue
Daniel Martí [Fri, 14 Aug 2015 22:30:03 +0000 (15:30 -0700)]
Also enforce https on urls without path
Daniel Martí [Fri, 14 Aug 2015 21:25:06 +0000 (14:25 -0700)]
Specify that CVC 0 means infinity
Daniel Martí [Fri, 14 Aug 2015 21:16:43 +0000 (14:16 -0700)]
lint: warn about CVCs that don't make sense
Daniel Martí [Fri, 14 Aug 2015 17:30:27 +0000 (10:30 -0700)]
Line numbers start at 1, not 0
Daniel Martí [Fri, 14 Aug 2015 17:27:16 +0000 (10:27 -0700)]
Add "Update Check Name:Ignore"
Daniel Martí [Fri, 14 Aug 2015 05:37:42 +0000 (22:37 -0700)]
lint: Fix unicode issues in list warnings
Daniel Martí [Fri, 14 Aug 2015 05:14:59 +0000 (22:14 -0700)]
lint: don't warn multiple times about lists
Daniel Martí [Fri, 14 Aug 2015 05:13:58 +0000 (22:13 -0700)]
lint: warn about incorrect lists
Daniel Martí [Thu, 13 Aug 2015 19:44:35 +0000 (12:44 -0700)]
Fix test that broke with
c813a3cdbb7e
Daniel Martí [Thu, 13 Aug 2015 18:28:01 +0000 (11:28 -0700)]
Add gradle 2.5, fixes #108
Daniel Martí [Thu, 13 Aug 2015 17:45:43 +0000 (10:45 -0700)]
Always end config list items with a comma
Ciaran Gultnieks [Thu, 13 Aug 2015 17:16:28 +0000 (19:16 +0200)]
Fix multiple errors in get_mime_type
Closes #111
Daniel Martí [Sun, 9 Aug 2015 01:40:20 +0000 (01:40 +0000)]
Merge branch 'fix-custom-buildToolsVersion' into 'master'
Fix gradle adaptation for custom buildToolsVersion configurations
I have ```build_tools = "22.0.1"``` configured in my config.py, however a recent fdroidserver update broke this. This PR fixes it for me.
See merge request !64
Lode Hoste [Sat, 8 Aug 2015 19:29:48 +0000 (21:29 +0200)]
fix gradle adaptation for custom buildToolsVersion configurations
Hans-Christoph Steiner [Fri, 7 Aug 2015 18:15:37 +0000 (18:15 +0000)]
Merge branch 'master' into 'master'
Add configuration for pypi
See: [http://peterdowns.com/posts/first-time-with-pypi.html](http://peterdowns.com/posts/first-time-with-pypi.html)
See merge request !63
Nico Alt [Fri, 7 Aug 2015 09:41:38 +0000 (11:41 +0200)]
Add configuration for pypi
Daniel Martí [Wed, 5 Aug 2015 17:52:53 +0000 (10:52 -0700)]
lint: complain about UCMs that haven't been used
Mainly for MRs from people who don't fill the automatic fields
Daniel Martí [Wed, 5 Aug 2015 17:33:18 +0000 (10:33 -0700)]
Properly default to NDK r10e
Daniel Martí [Wed, 5 Aug 2015 16:38:27 +0000 (16:38 +0000)]
Merge branch 'master' into 'master'
final 0.4.0 fixes
Because `setup.py` is broken because `MANIFEST.in` is still pointing to `README` and not `README.md`, the 0.4.0 tag needs to be deleted. This merge requests fixes that and two other minor issues. This has passed multiple runs of the test suite, both on my machine and the jenkins box.
See merge request !61
Hans-Christoph Steiner [Wed, 5 Aug 2015 11:48:42 +0000 (13:48 +0200)]
fix `fdroid rewritemeta` reference in manual
Hans-Christoph Steiner [Wed, 5 Aug 2015 11:26:35 +0000 (13:26 +0200)]
run-tests: fix quoting to prevent spurious execution of `fdroid update`
Hans-Christoph Steiner [Wed, 5 Aug 2015 08:35:14 +0000 (10:35 +0200)]
rename README to README.md in MANIFEST.in
this stops `python setup.py` from failing...
Daniel Martí [Wed, 5 Aug 2015 06:00:11 +0000 (23:00 -0700)]
Bump to 0.4.0
Hans-Christoph Steiner [Tue, 4 Aug 2015 20:56:16 +0000 (20:56 +0000)]
Merge branch 'master' into 'master'
Port README to markdown
Also add instructions for Arch-Linux.
You can take a look on it [here](https://gitlab.com/NicoAlt/fdroidserver/blob/master/README.md).
See merge request !60
Nico Alt [Tue, 4 Aug 2015 19:02:06 +0000 (21:02 +0200)]
Correct filename in setup.py
Nico Alt [Tue, 4 Aug 2015 10:54:06 +0000 (12:54 +0200)]
Port README to markdown
Also add instructions for Arch-Linux.