chiark / gitweb /
Hans-Christoph Steiner [Wed, 19 Jul 2017 14:08:21 +0000 (16:08 +0200)]
update README to point to https://f-droid.org/docs
Hans-Christoph Steiner [Wed, 19 Jul 2017 14:03:50 +0000 (16:03 +0200)]
update file list in setup.py
Hans-Christoph Steiner [Wed, 26 Jul 2017 05:06:42 +0000 (05:06 +0000)]
Merge branch 'metadata' into 'master'
Updating syntax check for Flattr (see #284)
See merge request !309
Izzy [Thu, 20 Jul 2017 18:26:25 +0000 (20:26 +0200)]
Updating syntax check for Flattr (see #284)
Hans-Christoph Steiner [Thu, 20 Jul 2017 03:58:27 +0000 (03:58 +0000)]
Merge branch '0.8-fixes' into 'master'
0.8 fixes
Closes #325, #347, and #309
See merge request !304
Hans-Christoph Steiner [Wed, 19 Jul 2017 21:59:47 +0000 (23:59 +0200)]
Hans-Christoph Steiner [Wed, 19 Jul 2017 21:58:07 +0000 (23:58 +0200)]
include all test files in source tarball
Hans-Christoph Steiner [Wed, 19 Jul 2017 13:56:50 +0000 (15:56 +0200)]
gitlab-ci: filter new build fields in metadata_v0 test
Hans-Christoph Steiner [Wed, 19 Jul 2017 08:59:05 +0000 (10:59 +0200)]
update: always include name/summary/desc in index.xml if available
With the new localization support, the name/summary/description in the
metadata file becomes the global override. So most apps are not going to
have those fields present in their metadata file. This fixes the index.xml
generation to fall back to the localized versions of those fields when they
are not set in the metadata field.
https://forum.f-droid.org/t/what-has-happend-to-osmand
Hans-Christoph Steiner [Wed, 19 Jul 2017 07:56:28 +0000 (09:56 +0200)]
add new files to MANIFEST.in
They need to be there in order to be included in the source tarball.
Hans-Christoph Steiner [Wed, 19 Jul 2017 07:55:45 +0000 (09:55 +0200)]
wp-droid: move deprecated Wordpress plugin to its own repo
closes #325
Hans-Christoph Steiner [Mon, 17 Jul 2017 10:11:33 +0000 (12:11 +0200)]
verify: if downloading from /repo/ fails, try /archive/
The Builds entries in metadata/ files do not easily say whether a
given APK is in the repo/ or the archive/. So it should also try to
download the official APK from the archive/ when verifying.
Hans-Christoph Steiner [Sat, 15 Jul 2017 20:15:30 +0000 (20:15 +0000)]
keep .apk file ext when diffoscope'ing Binaries:
Hans-Christoph Steiner [Sat, 15 Jul 2017 21:42:29 +0000 (23:42 +0200)]
rewritemeta: only print file type if its changing
Hans-Christoph Steiner [Wed, 19 Jul 2017 11:02:06 +0000 (13:02 +0200)]
server: report errors pushing to git mirrors
This makes `fdroid server update` fail if pushing to one of the git mirrors
fails. This is what happens if the other methods fail, e.g. rsync or S3.
closes #347
Hans-Christoph Steiner [Wed, 19 Jul 2017 10:59:20 +0000 (12:59 +0200)]
server: include gitlab raw URLs as git mirrors
gitlab serves raw files from a CDN, so its appropriate to use the raw URL.
@pserwylo @grote and I discussed it and found a reference, but I can't find
that reference now.
Since the client will try the next mirror if one fails, it makes sense to
include both the gitlab raw and gitlab pages URLs to the mirror. The
gitlab pages deploy process is still a bit flaky anyway.
Hans-Christoph Steiner [Mon, 10 Jul 2017 15:13:26 +0000 (17:13 +0200)]
server: only rm git mirror if the git history is getting too large
git hosts like github, gitlab, bitbucket usually allow 1 gig repos. This
changes the git mirroring behavior to keep the history until the repo hits
1 gig. Keeping history makes updates a lot faster, since the whole repo
does not need to be pushed on each update.
Hans-Christoph Steiner [Sat, 8 Jul 2017 10:11:42 +0000 (12:11 +0200)]
update: force checkout .gitlab-ci.yml when updating git mirrors
closes #309
Hans-Christoph Steiner [Fri, 7 Jul 2017 15:52:53 +0000 (17:52 +0200)]
server: smooth out btlog transfer for offline signing setups
It turns out it is error prone to `git push` to a non-bare git repo. For
the offline signing machine, the git remote needs to be a regular git repo
in a directory on a thumbdrive so that once the thumbdrive is plugged into
an online machine, that git repo can be transferred to the online machine.
Hans-Christoph Steiner [Mon, 17 Jul 2017 06:33:02 +0000 (06:33 +0000)]
Merge branch 'xmlicons' into 'master'
have fallback for XML icons also consider res/mipmap*
See merge request !307
Izzy [Sun, 16 Jul 2017 22:54:33 +0000 (00:54 +0200)]
have fallback for XML icons also consider res/mipmap*
several apps (e.g.
[FastHub](http://apt.qumran.org/fdroid/index/apk/com.fastaccess.github)
and [Monety](http://apt.qumran.org/fdroid/index/apk/open.currency)) have
their ic_launcher.png files not in res/drawable*dpi/, but in
res/mipmap*/ -- so the regex has been adjusted by this patch.
Additionally: if the only icon for a given resolution was an XML without
existing fallback-PNG, it should be considered "non existent" (ie.
"empty_density").
Hans-Christoph Steiner [Sun, 16 Jul 2017 14:21:46 +0000 (14:21 +0000)]
Merge branch 'metadata' into 'master'
adding example metadata template (closes #345)
Closes #345
See merge request !306
Izzy [Sat, 15 Jul 2017 15:22:18 +0000 (17:22 +0200)]
using a "real category" as example
Izzy [Sat, 15 Jul 2017 14:52:08 +0000 (16:52 +0200)]
adding example metadata template (closes #345)
Hans-Christoph Steiner [Mon, 10 Jul 2017 09:08:45 +0000 (09:08 +0000)]
Merge branch 'metadata' into 'master'
--create-metadata: use yaml.dump() and (if exists) template.yml
See merge request !305
Izzy [Sat, 8 Jul 2017 12:41:19 +0000 (14:41 +0200)]
--create-metadata: only set default empty values if not using template.py
Izzy [Sat, 8 Jul 2017 12:21:49 +0000 (14:21 +0200)]
enable user to have presets for metadata by using a template.yml (see #345)
Izzy [Sat, 8 Jul 2017 12:07:11 +0000 (14:07 +0200)]
make --create-metadata use yaml.dump instead of ruamel (solves #345)
Hans-Christoph Steiner [Fri, 7 Jul 2017 13:50:11 +0000 (15:50 +0200)]
ensure that mirror URLs always include the repodir
Since the mirror URLs are per repo section (repo/archive), the mirror URLs
must include the repodir at the end. This was missing for servergitmirrors
found by @cde when working on fdroidclient#35
Hans-Christoph Steiner [Fri, 7 Jul 2017 11:02:32 +0000 (11:02 +0000)]
Merge branch 'master' into 'master'
Also search for apk in build/outputs/apk/release
See merge request !303
mimi89999 [Fri, 7 Jul 2017 10:21:09 +0000 (12:21 +0200)]
Also search for apk in build/outputs/apk/release
Hans-Christoph Steiner [Thu, 6 Jul 2017 22:49:26 +0000 (22:49 +0000)]
Merge branch 'metadata' into 'master'
--create-metadata: make sure apk[name] is not empty
See merge request !302
Izzy [Thu, 6 Jul 2017 22:18:08 +0000 (00:18 +0200)]
--create-metadata: make sure apk[name] is not empty
Hans-Christoph Steiner [Thu, 6 Jul 2017 21:53:31 +0000 (21:53 +0000)]
Merge branch 'sudo' into 'master'
add new 'sudo=' Build field
Closes #317
See merge request !297
Hans-Christoph Steiner [Wed, 28 Jun 2017 21:01:45 +0000 (23:01 +0200)]
support configing buildserver VM per-build with sudo=
This adds the 'sudo' build field, which is just a script that is run as
root. For more info, see the issue that this closes:
refs #318
closes #317
Michael Pöhn [Thu, 6 Jul 2017 15:29:19 +0000 (15:29 +0000)]
Merge branch 'yaml-template' into 'master'
update: remove ruamel requirement, and improve '--create-metadata'
Closes #343
See merge request !300
Hans-Christoph Steiner [Thu, 6 Jul 2017 09:24:55 +0000 (11:24 +0200)]
handle App instance with no Builds when writing YAML
Hans-Christoph Steiner [Thu, 6 Jul 2017 11:25:14 +0000 (13:25 +0200)]
update: update openssl KnownVuln scan to handle all recent versions
Thanks to @bubu for reporting!
Hans-Christoph Steiner [Thu, 6 Jul 2017 09:06:47 +0000 (11:06 +0200)]
update: remove ruamel requirement, and improve '--create-metadata'
If ruamel.yaml is not available, this will fallback to using PyYAML. This
also adds some blank fields to the newly created template to make it easy
for human editors to fill in.
closes #343
Hans-Christoph Steiner [Thu, 6 Jul 2017 11:29:36 +0000 (11:29 +0000)]
Merge branch 'some-data-related-fixed' into 'master'
support manually adding per-Build Anti-Features in metadata, and other fixes
Closes #322 and #331
See merge request !296
Hans-Christoph Steiner [Tue, 4 Jul 2017 15:40:02 +0000 (17:40 +0200)]
update: find PNG when recommended icon is an XML file
APKs can now use XML files for vector graphics like the app icon. `aapt`
returns the XML file by default, and perhaps also androguard. This
checks if the icon is an XML file, and if so, it tries to find a PNG in
the APK with the same name and density to use instead
closes #322
This should also ultimately make the XML file available as an icon source
as well fdroidclient#1091
Hans-Christoph Steiner [Tue, 4 Jul 2017 15:18:21 +0000 (17:18 +0200)]
update: normalize var name to apkzip
Hans-Christoph Steiner [Thu, 29 Jun 2017 14:00:16 +0000 (16:00 +0200)]
jenkins: document build/sign test flow
Hans-Christoph Steiner [Wed, 28 Jun 2017 14:55:34 +0000 (16:55 +0200)]
buildserver: support any recent NDK version, with stable filenames
Now that the download file name and type seems to have stabilized, I
think we no longer need to manually specify each new added release in
this script to unpack.
closes #331
Hans-Christoph Steiner [Tue, 27 Jun 2017 21:55:38 +0000 (23:55 +0200)]
support manually adding per-build antiFeatures in metadata
For cases like the OpenVPN vuln that was recently announced, it is useful
for fdroiddata maintainers to be able to mark builds that have known
vulnerabilities.
Michael Pöhn [Tue, 27 Jun 2017 20:46:19 +0000 (22:46 +0200)]
lint: check all build fields against accepted list
This makes sure there are no typoed or wrong build fields in all metadata
files.
Hans-Christoph Steiner [Thu, 6 Jul 2017 08:10:14 +0000 (08:10 +0000)]
Merge branch '343-ruamel-yaml-version-check' into 'master'
check version of ruamel.yaml and raise according error message
See merge request !301
Michael Pöhn [Thu, 6 Jul 2017 07:07:54 +0000 (09:07 +0200)]
check version of ruamel.yaml and raise according error message
Hans-Christoph Steiner [Wed, 5 Jul 2017 20:34:35 +0000 (20:34 +0000)]
Merge branch 'antifeature' into 'master'
added new AntiFeature: ApplicationDebuggable
See merge request !298
Hans-Christoph Steiner [Wed, 5 Jul 2017 20:31:23 +0000 (20:31 +0000)]
Merge branch 'metadata' into 'master'
create_metadata: do not skip APK files having no name
See merge request !299
Izzy [Wed, 5 Jul 2017 20:31:21 +0000 (20:31 +0000)]
create_metadata: do not skip APK files having no name
Izzy [Tue, 4 Jul 2017 19:15:04 +0000 (21:15 +0200)]
added new AntiFeature: ApplicationDebuggable
Useful e.g. for "test repositories" to indicate an app was compiled with
"application-debuggable"
Hans-Christoph Steiner [Tue, 4 Jul 2017 12:53:21 +0000 (14:53 +0200)]
gitlab-ci: remove hacks need to support Debian/jessie
Hans-Christoph Steiner [Tue, 4 Jul 2017 12:34:42 +0000 (14:34 +0200)]
Merge branch 'ndk' into 'master'
NDK r14b and r15b
See merge request !293
Hans-Christoph Steiner [Tue, 4 Jul 2017 12:24:17 +0000 (12:24 +0000)]
Merge branch '290-make-rewrite-metadata-respect-key-word-sort-order' into 'master'
Resolve "make `fdroid rewritemeta` respect key-word sort order for YAML files"
Closes #290
See merge request !295
Michael Pöhn [Tue, 4 Jul 2017 11:51:59 +0000 (11:51 +0000)]
Merge branch '290-make-rewrite-metadata-respect-key-word-sort-order' into 'master'
rewritemeta: respect key-word sort order for YAML files
Closes #290
See merge request !263
Michael Pöhn [Tue, 4 Jul 2017 11:35:05 +0000 (13:35 +0200)]
moved some yaml test files into a separate forlder to avoid conflicts with other test cases
Michael Pöhn [Tue, 4 Jul 2017 11:27:45 +0000 (13:27 +0200)]
correct yaml-rewriting for buildozer flag
Michael Pöhn [Tue, 23 May 2017 10:36:57 +0000 (12:36 +0200)]
import ruamel.ymal only when re-writing yaml metadata
Michael Pöhn [Fri, 19 May 2017 11:47:05 +0000 (13:47 +0200)]
rewritemeta yaml: fixed boolen mapping for build flags
Michael Pöhn [Tue, 16 May 2017 14:28:24 +0000 (16:28 +0200)]
use stored metadatapath instead of guessing it
Michael Pöhn [Tue, 16 May 2017 13:26:59 +0000 (15:26 +0200)]
fix indentation
Hans-Christoph Steiner [Fri, 25 Nov 2016 14:23:01 +0000 (15:23 +0100)]
`fdroid update --create-metadata` now outputs YAML format
As part of the push towards using YAML as the main metadata format, this
makes the blank template be a .yml file.
Michael Pöhn [Tue, 16 May 2017 13:04:37 +0000 (15:04 +0200)]
fix building with yml metadata
Michael Pöhn [Tue, 16 May 2017 08:29:32 +0000 (10:29 +0200)]
set required minimum ruamel.yaml version
Michael Pöhn [Tue, 9 May 2017 12:13:14 +0000 (14:13 +0200)]
refactored yaml tests into metadata test case; fixed typo
Michael Pöhn [Tue, 9 May 2017 11:15:28 +0000 (13:15 +0200)]
add ruamel yaml to buildserver depenencies
Michael Pöhn [Sun, 7 May 2017 16:23:22 +0000 (18:23 +0200)]
completed ordered yaml field list
Michael Pöhn [Sun, 7 May 2017 00:13:25 +0000 (02:13 +0200)]
prettify write_yaml metadata
Michael Pöhn [Tue, 2 May 2017 12:13:30 +0000 (14:13 +0200)]
yaml rewrite version code as int
Michael Pöhn [Tue, 2 May 2017 11:00:33 +0000 (13:00 +0200)]
rewrite metadata builds list to yaml
Michael Pöhn [Tue, 2 May 2017 08:06:42 +0000 (10:06 +0200)]
yaml metadata rewrite: correct blank line
Michael Pöhn [Mon, 1 May 2017 20:40:14 +0000 (22:40 +0200)]
add ruamel.yaml to setup.py
Michael Pöhn [Mon, 1 May 2017 20:35:06 +0000 (22:35 +0200)]
fixed pyflakes/pep warings
Michael Pöhn [Mon, 1 May 2017 19:19:51 +0000 (21:19 +0200)]
rewrite to yaml works for app data now (builds still missing)
relan [Tue, 4 Jul 2017 08:44:25 +0000 (11:44 +0300)]
makebuildserver: add NDK r15b
relan [Tue, 4 Jul 2017 07:46:52 +0000 (10:46 +0300)]
makebuildserver: fix NDK r14b provisioning
Replace r14 with r14b after
a57bff7.
Hans-Christoph Steiner [Mon, 3 Jul 2017 22:08:43 +0000 (22:08 +0000)]
Merge branch 'repo-update-break-down' into 'master'
Break down the update.scan_apk() method into smaller pieces
See merge request !288
Torsten Grote [Wed, 14 Jun 2017 14:12:25 +0000 (11:12 -0300)]
Break up the scan_apk() method and rename it to process_apk()
Hans-Christoph Steiner [Mon, 3 Jul 2017 18:40:15 +0000 (20:40 +0200)]
gitlab-ci: switch to new Debian/stretch image
ci-images-server!1
Hans-Christoph Steiner [Mon, 3 Jul 2017 09:07:08 +0000 (09:07 +0000)]
Merge branch 'archive-policy-fix' into 'master'
Archive policy overhaul
Closes #323, #292, and #166
See merge request !291
Hans-Christoph Steiner [Thu, 29 Jun 2017 17:53:15 +0000 (19:53 +0200)]
gitlab-ci: apt upgrade so that tests run with current updates
The MD5 signature stuff was failing in tests because the CI image was
using a quite old version of Java's jarsigner, which had not yet disabled
MD5.
Hans-Christoph Steiner [Thu, 29 Jun 2017 19:15:30 +0000 (21:15 +0200)]
update: invalidate cache if allow_disabled_algorithms changes
Since the cache contains implicitly the result of the jarsigner verify,
if the allow_disabled_algorithms config changes, then the apkcache is
invalid.
Hans-Christoph Steiner [Thu, 29 Jun 2017 18:28:16 +0000 (20:28 +0200)]
update: write cache file if anything has changed it
This fixes a bug introduced in
04db6870 where cachechanged for scan_apks()
was set only by the last call to scan_apk().
Hans-Christoph Steiner [Wed, 28 Jun 2017 20:10:43 +0000 (22:10 +0200)]
tests: make sure apkcache gets created
Hans-Christoph Steiner [Tue, 27 Jun 2017 20:07:53 +0000 (22:07 +0200)]
update: create 'archive/' if needed when moving APKs
Normally, just 'repo/' is created by default, e.g. `fdroid init`. If APKs
are dumped into 'repo/', then have invalid signatures, then they'll be
automatically moved to 'archive/', which therefore needs to exist.
Hans-Christoph Steiner [Tue, 27 Jun 2017 19:40:39 +0000 (21:40 +0200)]
update: allow_disabled_algorithms option to keep MD5 sigs in repo
The new policy is to move APKs with invalid signatures to the archive,
and only add those APKs to the archive's index if they have valid MD5
signatures.
closes #323
closes #292
Hans-Christoph Steiner [Tue, 27 Jun 2017 07:54:35 +0000 (09:54 +0200)]
update: allow deprecated signatures only in the archive
In April 2017, Oracle's jarsigner and Google's apksigner both switched to
considering any APK signature that uses MD5 as unsigned. Any old build
is likely to have a MD5 signature. This sets up the archive as the only
place where these "disabled algorithms" are allowed in the repo, and
marks any APK signed by a "disabled algorithm" as having a "known
vulnerability"
This also now automatically moves APKs with invalid signatures to the
archive section.
#323
Hans-Christoph Steiner [Mon, 26 Jun 2017 19:08:01 +0000 (21:08 +0200)]
update: move duplicated code into move_apk_between_sections()
Hans-Christoph Steiner [Fri, 23 Jun 2017 21:55:12 +0000 (23:55 +0200)]
fix "Archive Policy:" field, APKs can move in/out of archive
The original logic was checking keepversions against the len() of ALL the
APKs in the repo/archive. The correct thing is to check against the
number of APKs available for the given packageName/appid.
closes #166
Hans-Christoph Steiner [Tue, 27 Jun 2017 21:33:24 +0000 (23:33 +0200)]
tests: test moving files to and from the archive
#166
Hans-Christoph Steiner [Wed, 28 Jun 2017 21:53:18 +0000 (21:53 +0000)]
Merge branch 'random-small-fixes' into 'master'
Random small fixes
Closes #222
See merge request !292
Hans-Christoph Steiner [Wed, 28 Jun 2017 20:23:04 +0000 (22:23 +0200)]
init: update docs links for next steps
Hans-Christoph Steiner [Fri, 23 Jun 2017 19:58:46 +0000 (21:58 +0200)]
lint: check file extension for metadata
This helps keep fdroiddata clean, on @krt's request.
closes #222
Hans-Christoph Steiner [Fri, 23 Jun 2017 14:38:40 +0000 (16:38 +0200)]
locale: fix broken source string
Hans-Christoph Steiner [Wed, 21 Jun 2017 12:01:01 +0000 (14:01 +0200)]
scan APKs for signs of "Master Key" exploit
This exploit is old, and was fixed in 4.4. But it was easy to exploit,
so it is still worth scanning for it. It is also easy to scan for, since
valid APKs should not have files with duplicate names. In theory, this
could look for duplicate file names for any file, but this limits the
false positives by only checking names of files related to executing code.
fdroidclient#40
Hans-Christoph Steiner [Thu, 15 Jun 2017 15:02:46 +0000 (17:02 +0200)]
update: move btlog import since btlog.py requires python3-git
To keep the dependencies limited to where they are needed.
Hans-Christoph Steiner [Wed, 28 Jun 2017 21:12:04 +0000 (23:12 +0200)]
gitlab-ci: fix metadata_v0 test to run on the right commits
Checking out master will often mean its testing the wrong commit, since
merge requests rarely are in master.
Hans-Christoph Steiner [Thu, 22 Jun 2017 14:32:44 +0000 (16:32 +0200)]
gitlab-ci: fix metadata_v0 test for new buildozer build flag
Just remove it, since 0.7.0 does not know about that build flag.
Hans-Christoph Steiner [Wed, 21 Jun 2017 12:04:45 +0000 (12:04 +0000)]
Merge branch 'buildozer-build' into 'master'
Added a method to build python/kivy projects using buildozer.
See merge request !290