@copying
This manual is for the F-Droid repository server tools.
-Copyright @copyright{} 2010, 2011, 2012, 2013 Ciaran Gultnieks
+Copyright @copyright{} 2010, 2011, 2012, 2013, 2014, 2015 Ciaran Gultnieks
Copyright @copyright{} 2011 Henrik Tunedal, Michael Haas, John Sullivan
Copyright @copyright{} 2013 David Black
-Copyright @copyright{} 2013, 2014 Daniel Martí
+Copyright @copyright{} 2013, 2014, 2015 Daniel Martí
+
+Copyright @copyright{} 2015 Boris Kraut
@quotation
Permission is granted to copy, distribute and/or modify this document
@item
GNU/Linux
@item
-Python 2.x
+Python 3.4 or later
@item
The Android SDK Tools and Build-tools.
Note that F-Droid does not assume that you have the Android SDK in your
@item
Ruby (debian packages ruby and rubygems)
@item
-Vagrant (unpackaged) Be sure to use 1.3.x because 1.4.x is completely broken
-(at the time of writing, the forthcoming 1.4.3 might work)
+Vagrant (debian package vagrant - 1.4.x or higher required)
+@item
+vagrant-cachier plugin (unpackaged): `vagrant plugin install vagrant-cachier`
@item
Paramiko (debian package python-paramiko)
@item
Imaging (debian package python-imaging)
-@item
-Magic (debian package python-magic)
@end itemize
On the other hand, if you want to build the apps directly on your system
Regardless of the intended usage of the tools, you will always need to set
up some basic configuration details. This is done by creating a file called
@code{config.py} in the data directory. You should do this by copying the
-example file (@code{config.sample.py}) from the fdroidserver project to your
-data directory and then editing according to the instructions within.
+example file (@code{./examples/config.py}) from the fdroidserver project to
+your data directory and then editing according to the instructions within.
Once configured in this way, all the functionality of the tools is accessed
by running the @code{fdroid} command. Run it on its own to get a list of the
@item
Make a directory for your repository. This is the directory from which you
will do all the work with your repository. Create a config file there, called
-@code{config.py}, by copying the @code{config.sample.py} from the server
+@code{config.py}, by copying @code{./examples/config.py} from the server
project and editing it.
@item
Within that, make a directory called @code{repo} and put APK files in it.
following:
@example
-./fdroid build org.fdroid.fdroid:16
+fdroid build org.fdroid.fdroid:16
@end example
This attempts to build version code 16 (which is version 0.25) of the F-Droid
If you were intending to publish these files, you could then run:
@example
-./fdroid publish
+fdroid publish
@end example
The source tarball would move to the @code{repo} directory (which is the
party.
+@section Running "fdroid build" in your app's source
+
+Another option for using @code{fdroid build} is to use a metadata file
+that is included in the app's source itself, rather than in a
+@code{metadata/} folder with lots of other apps. This metadata file
+should be in the root of your source repo, and be called
+@code{.fdroid.json}, @code{.fdroid.xml}, @code{.fdroid.yaml}, or
+@code{.fdroid.txt}, depending on your preferred data format: JSON,
+XML, YAML, or F-Droid's @code{.txt} format.
+
+Once you have that setup, you can build the most recent version of
+the app using the whole FDroid stack by running:
+
+@example
+fdroid build
+@end example
+
+If you want to build every single version, then specify @code{--all}.
+
+
@section Direct Installation
You can also build and install directly to a connected device or emulator
@node Importing Applications
@chapter Importing Applications
-To help with starting work on including a new application, @code{fdroid import}
-will take a URL and optionally some other parameters, and attempt to construct
-as much information as possible by analysing the source code. Basic usage is:
+To help with starting work on including a new application, use
+@code{fdroid import} to set up a new template project. It has two
+modes of operation, starting with a cloned git repo:
@example
-./fdroid import --url=http://address.of.project
+git clone https://gitlab.com/fdroid/fdroidclient
+cd fdroidclient
+fdroid import
@end example
-For this to work, the URL must point to a project format that the script
+Or starting with a URL to a project page:
+
+@example
+fdroid import --url=http://address.of.project
+@end example
+
+When a URL is specified using the @code{--url=} flag, @code{fdroid
+import} will use that URL to find out information about the project,
+and if it finds a git repo, it will also clone that. For this to
+work, the URL must point to a project format that the script
understands. Currently this is limited to one of the following:
@enumerate
@item
+GitLab - @code{https://gitlab.com/PROJECTNAME/REPONAME}
+@item
Gitorious - @code{https://gitorious.org/PROJECTNAME/REPONAME}
@item
Github - @code{https://github.com/USER/PROJECT}
the metadata files in the metadata directory.
@end enumerate
-The metadata files are simple, easy to edit text files, always named as the
-application's package ID with '.txt' appended.
+The original metadata files are simple, easy to edit text files,
+always named as the application's package ID with '.txt' appended.
+Additionally, you can use JSON, XML, or YAML for app metadata, using
+the same fields as the original '.txt' format.
+
+Note that although the metadata files are designed to be easily read
+and writable by humans, they are also processed and written by various
+scripts. The original '.txt' format can be automatically cleaned up
+when necessary. The structure and comments will be preserved
+correctly, although the order of fields will be standardised. (In the
+event that the original file was in a different order, comments are
+considered as being attached to the field following them). In fact,
+you can standardise all the '.txt' metadata in a single command,
+without changing the functional content, by running:
-Note that although the metadata files are designed to be easily read and
-writable by humans, they are also processed and written by various scripts.
-They are capable of rewriting the entire file when necessary. Even so,
-the structure and comments will be preserved correctly, although the order
-of fields will be standardised. (In the event that the original file was
-in a different order, comments are considered as being attached to the field
-following them). In fact, you can standardise all the metadata in a single
-command, without changing the functional content, by running:
+@example
+fdroid rewritemeta
+@end example
+
+Or just run it on a specific app:
@example
-fdroid rewritemetadata
+fdroid rewritemeta org.adaway
@end example
The following sections describe the fields recognised within the file.
@menu
* Categories::
+* Author Name::
+* Author Email::
* License::
* Auto Name::
* Name::
* Web Site::
* Source Code::
* Issue Tracker::
+* Changelog::
* Donate::
* FlattrID::
* Bitcoin::
* Maintainer Notes::
* Repo Type::
* Repo::
+* Binaries::
* Build::
* AntiFeatures::
* Disabled::
* Update Check Mode::
* Update Check Ignore::
* Vercode Operation::
+* Update Check Name::
* Update Check Data::
* Auto Update Mode::
* Current Version::
This is converted to (@code{<categories>}) in the public index file.
+@node Author Name
+@section Author Name
+
+@cindex Author Name
+
+The name of the author, either full, abbreviated or pseudonym. If
+present, it should represent the name(s) as published by upstream,
+e.g. in their copyright or authors file. This can be omitted (or left
+blank).
+
+This is converted to (@code{<author>}) in the public index file.
+
+@node Author Email
+@section Author Email
+
+@cindex Author Email
+
+The e-mail address of the author(s). This can be omitted (or left
+blank).
+
+This is converted to (@code{<email>}) in the public index file.
+
@node License
@section License
This is converted to (@code{<tracker>}) in the public index file.
+@node Changelog
+@section Changelog
+
+@cindex Changelog
+
+The URL for the application's changelog. Optional, since not all
+applications have one.
+
+This is converted to (@code{<changelog>}) in the public index file.
+
@node Donate
@section Donate
@cindex Summary
A brief summary of what the application is. Since the summary is only allowed
-one line on the list of the F-Droid client, keeping it to within 50 characters
+one line on the list of the F-Droid client, keeping it to within 80 characters
will ensure it fits most screens.
@node Description
@item
@samp{srclib}
@end itemize
+
@node Repo
@section Repo
Here's an example of a complex git-svn Repo URL:
http://svn.code.sf.net/p/project/code/svn;trunk=trunk;tags=tags;branches=branches
-For a Subversion repo that requires authentication, you can precede the repo
-URL with username:password@ and those parameters will be passed as @option{--username}
-and @option{--password} to the SVN checkout command. (This now works for both
-svn and git-svn)
-
If the Repo Type is @code{srclib}, then you must specify the name of the
according srclib .txt file. For example if @code{scrlibs/FooBar.txt} exist
and you want to use this srclib, then you have to set Repo to
@code{FooBar}.
+@node Binaries
+@section Binaries
+
+@cindex Binaries
+
+The location of binaries used in verification process.
+
+If specified, F-Droid will verify the output apk file of a build against the
+one specified. You can use %v and %c to point to the version name and version
+code of the current build. To verify the F-Droid client itself you could use:
+@code{Binaries:https://f-droid.org/repo/org.fdroid.fdroid_%c.apk}
+
+F-Droid will use upstream binaries if the verification succeeded.
+
@node Build
@section Build
takes place.
You can use $$SDK$$, $$NDK$$ and $$MVN3$$ to substitute the paths to the
-android SDK and NDK directories, and maven 3 executable respectively.
+android SDK and NDK directories, and maven 3 executable respectively. The
+following per-build variables are available likewise: $$VERSION$$,
+$$VERCODE$$ and $$COMMIT$$.
@item oldsdkloc=yes
The sdk location in the repo is in an old format, or the build.xml is
@item target=<target>
Specifies a particular SDK target for compilation, overriding the value
defined in the code by upstream. This has different effects depending on what
-build system used — this flag currently affects ant, maven and gradle projects
+build system used — this flag currently affects Ant, Maven and Gradle projects
only. Note that this does not change the target SDK in the
AndroidManifest.xml, which determines the level of features that can be
included in the build.
-In the case of an ant project, it modifies project.properties of the app and
+In the case of an Ant project, it modifies project.properties of the app and
possibly sub-projects. This is likely to cause the whole build.xml to be
rewritten, which is fine if it's a 'standard' android file or doesn't already
exist, but not a good idea if it's heavily customised.
@item update=<auto/dirs>
-By default, 'android update project' is used to generate or update the
+By default, 'android update' is used in Ant builds to generate or update the
project and all its referenced projects. Specifying update=no bypasses that.
-Note that this only matters in ant build recipes.
+Note that this is useless in builds that don't use Ant.
-Default value is '@code{auto}', which uses the paths used in the
-project.properties file to find out what project paths to update.
+Default value is '@code{auto}', which recursively uses the paths in
+project.properties to find all the subprojects to update.
-Otherwise, value can be a comma-separated list of directories in
-which to run 'android update project' relative to the main
-application directory (which may include '@code{subdir}' parameter).
+Otherwise, the value can be a comma-separated list of directories in which to
+run 'android update' relative to the application directory.
@item encoding=xxxx
Adds a java.encoding property to local.properties with the given
If specified, the package version code in the AndroidManifest.xml is
replaced with the version code for the build. See also forceversion.
-@item rm=relpath1,relpath2,...
+@item rm=<path1>[,<path2>,...]
Specifies the relative paths of files or directories to delete before
the build is done. The paths are relative to the base of the build
directory - i.e. the root of the directory structure checked out from
Multiple files/directories can be specified by separating them with ','.
Directories will be recursively deleted.
-@item extlibs=a,b,...
+@item extlibs=<lib1>[,<lib2>,...]
Comma-separated list of external libraries (jar files) from the
@code{build/extlib} library, which will be placed in the @code{libs} directory
of the project.
the form name@@rev where name is the predefined source library name and rev is
the revision or tag to use in the respective source control.
-For ant projects, you can optionally append a number with a colon at the
+For Ant projects, you can optionally append a number with a colon at the
beginning of a srclib item to automatically place it in project.properties as
a library under the specified number. For example, if you specify
-@code{1:somelib@@1.0}, f-droid will automatically do the equivalent of the
+@code{1:somelib@@1.0}, F-Droid will automatically do the equivalent of the
legacy practice @code{prebuild=echo "android.library.reference.1=$$somelib$$"
>> project.properties}.
the init/prebuild/build command to substitute the relative path to the library
directory, but it could need tweaking if you've changed into another directory.
+Currently srclibs are necessary when upstream uses jar files or pulls
+dependencies from non-trusted repositories. While there is no guarantee that
+those binaries are free and correspondent to the source code, F-Droid allows
+the following known repositories until a source-built alternative is available:
+
+@itemize @bullet
+
+@item
+@samp{mavenCentral} - the original repo, hardcoded in Maven and Gradle.
+
+@item
+@samp{jCenter} - hardcoded in Gradle, this repo by Bintray tries to provide
+easier handling. It should sync with mavenCentral from time to time.
+
+@item
+@samp{OSS Sonatype} - maintained by the people behind mavenCentral, this
+repository focuses on hosting services for open source project binaries.
+
+@item
+@samp{JitPack.io} - builds directly from Github repositories. However,
+they do not provide any option to reproduce or verify the resulting
+binaries. Builds pre-release versions in some cases.
+
+@item
+@samp{Clojars} - Clojure libraries repo.
+
+@item
+@samp{CommonsWare} - repo holding a collection of open-source libs.
+
+@end itemize
+
@item patch=x
Apply patch(es). 'x' names one (or more - comma-seperated) files within a
directory below the metadata, with the same name as the metadata file but
the @code{srclib} directory for details of this.
You can use $$SDK$$, $$NDK$$ and $$MVN3$$ to substitute the paths to the
-android SDK and NDK directories, and maven 3 executable respectively e.g.
-for when you need to run @code{android update project} explicitly.
+android SDK and NDK directories, and Maven 3 executable respectively e.g.
+for when you need to run @code{android update project} explicitly. The
+following per-build variables are available likewise: $$VERSION$$, $$VERCODE$$
+and $$COMMIT$$.
-@item scanignore=path1,path2,...
+@item scanignore=<path1>[,<path2>,...]
Enables one or more files/paths to be excluded from the scan process.
This should only be used where there is a very good reason, and
probably accompanied by a comment explaining why it is necessary.
When scanning the source tree for problems, matching files whose relative
paths start with any of the paths given here are ignored.
-@item scandelete=path1,path2,...
-Similar to scanignore=, but instead of ignoring files under the given paths,
-it tells f-droid to delete the matching files directly.
+@item scandelete=<path1>[,<path2>,...]
+When running the scan process, any files that trigger errors - like binaries -
+will be removed. It acts just like scanignore=, but instead of ignoring the
+files, it removes them.
+
+Useful when a source code repository includes binaries or other unwanted files
+which are not needed for the build. Instead of removing them manually via rm=,
+using scandelete= is easier.
@item build=xxxx
As for 'prebuild', but runs during the actual build phase (but before the
-main ant/maven build). Use this only for actions that do actual building.
+main Ant/Maven build). Use this only for actions that do actual building.
Any prepartion of the source code should be done using 'init' or 'prebuild'.
-Any building that takes place before build= will be ignored, as either ant,
+Any building that takes place before build= will be ignored, as either Ant,
mvn or gradle will be executed to clean the build environment right before
build= (or the final build) is run.
You can use $$SDK$$, $$NDK$$ and $$MVN3$$ to substitute the paths to the
-android SDK and NDK directories, and maven 3 executable respectively.
+android SDK and NDK directories, and maven 3 executable respectively. The
+following per-build variables are available likewise: $$VERSION$$,
+$$VERCODE$$ and $$COMMIT$$.
@item buildjni=[yes|no|<dir list>]
Enables building of native code via the ndk-build script before doing
-the main ant build. The value may be a list of directories relative
+the main Ant build. The value may be a list of directories relative
to the main application directory in which to run ndk-build, or 'yes'
which corresponds to '.' . Using explicit list may be useful to build
multi-component projects.
The build and scan processes will complain (refuse to build) if this
parameter is not defined, but there is a @code{jni} directory present.
-If the native code is being built by other means like a gradle task, you
+If the native code is being built by other means like a Gradle task, you
can specify @code{no} here to avoid that. However, if the native code is
actually not required or used, remove the directory instead (using
@code{rm=jni} for example). Using @code{buildjni=no} when the jni code
isn't used nor built will result in an error saying that native
libraries were expected in the resulting package.
-@item gradle=<flavour>[@@<dir>]
-Build with gradle instead of ant, specifying what flavour to assemble.
-If <flavour> is 'yes', 'main' or empty, no flavour will be used. Note
-that this will not work on projects with flavours, since it will build
-all flavours and there will be no 'main' build.
-If @@<dir> is attached to <flavour>, then the gradle tasks will be run
-in that directory. This might be necessary if gradle needs to be run in
-the parent directory, in which case one would use
-'gradle=<flavour>@@..'.
+@item ndk=<version>
+Version of the NDK to use in this build. Defaults to the latest NDK release
+that included legacy toolchains, so as to not break builds that require
+toolchains no longer included in current versions of the NDK.
+
+The buildserver supports r9b with its legacy toolchains, r10e and the
+latest release as of writing this document, r12b. You may add support
+for more versions by adding them to 'ndk_paths' in your config file.
+
+@item gradle=<flavour1>[,<flavour2>,...]
+Build with Gradle instead of Ant, specifying what flavours to use. Flavours
+are case sensitive since the path to the output apk is as well.
+
+If only one flavour is given and it is 'yes', no flavour will be used.
+Note that for projects with flavours, you must specify at least one
+valid flavour since 'yes' will build all of them separately.
@item maven=yes[@@<dir>]
-Build with maven instead of ant. Like gradle, an extra @@<dir> tells f-droid
-to run maven inside that relative subdirectory.
+Build with Maven instead of Ant. An extra @@<dir> tells F-Droid to run Maven
+inside that relative subdirectory. Sometimes it is needed to use @@.. so that
+builds happen correctly.
+
+@item preassemble=<task1>[,<task2>,...]
+List of Gradle tasks to be run before the assemble task in a Gradle project
+build.
+
+@item gradleprops=<prop1>[,<prop2>,...]
+List of Gradle properties to pass via the command line to Gradle. A property
+can be of the form @code{foo} or of the form @code{key=value}.
-@item preassemble=<task1> <task2>
-Space-separated list of gradle tasks to be run before the assemble task
-in a gradle project build.
+For example: @code{gradleprops=enableFoo,someSetting=bar} will result in
+@code{gradle -PenableFoo -PsomeSetting=bar}.
-@item antcommand=xxx
-Specify an alternate ant command (target) instead of the default
+@item antcommands=<target1>[,<target2>,...]
+Specify an alternate set of Ant commands (target) instead of the default
'release'. It can't be given any flags, such as the path to a build.xml.
-@item output=path/to/output.apk
-To be used when app is built with a tool other than the ones natively
-supported, like GNU Make. The given path will be where the build= set of
-commands should produce the final unsigned release apk.
+@item output=glob/to/output.apk
+Specify a glob path where the resulting unsigned release apk from the
+build should be. This can be used in combination with build methods like
+@code{gradle=yes} or @code{maven=yes}, but if no build method is
+specified, the build is manual. You should run your build commands, such
+as @code{make}, in @code{build=}.
@item novcheck=yes
Don't check that the version name and code in the resulting apk are
This is optional - if present, it contains a comma-separated list of any of
the following values, describing an anti-feature the application has.
-Even though such apps won't be displayed unless a settings box is ticked,
-it is a good idea to mention the reasons for the anti-feature(s) in the
+It is a good idea to mention the reasons for the anti-feature(s) in the
description:
@itemize @bullet
without major changes to the app.
@item
-@samp{NonFreeAdd} - the application promotes non-Free add-ons, such that the
+@samp{NonFreeAdd} - the application promotes non-free add-ons, such that the
app is effectively an advert for other non-free software and such software is
not clearly labelled as such.
@item
-@samp{NonFreeDep} - the application depends on a non-Free application (e.g.
+@samp{NonFreeDep} - the application depends on a non-free application (e.g.
Google Maps) - i.e. it requires it to be installed on the device, but does not
include it.
+@item
+@samp{UpstreamNonFree} - the application is or depends on non-free software.
+This does not mean that non-free software is included with the app: Most
+likely, it has been patched in some way to remove the non-free code. However,
+functionality may be missing.
+
+@item
+@samp{NonFreeAssets} - the application contains and makes use of non-free
+assets. The most common case is apps using artwork - images, sounds, music,
+etc - under a non-commercial license.
+
@end itemize
@node Disabled
archive. This app-specific policy setting can override that.
Currently the only supported format is "n versions", where n is the number
-of versions to keep.
+of versions to keep. Defaults to "3 versions".
@node Update Check Mode
@section Update Check Mode
of doing so. Updates should be checked for manually. Use this, for example,
when deploying betas or patched versions; when builds are done in a directory
different to where the AndroidManifest.xml is; if the developers use the
-gradle build system and store version info in a separate file; if the
+Gradle build system and store version info in a separate file; if the
developers make a new branch for each release and don't make tags; or if you've
changed the package name or version code logic.
@item
@node Update Check Ignore
@section Update Check Ignore
+@cindex Update Check Ignore
+
When checking for updates (via @code{Update Check Mode}) this can be used to
specify a regex which, if matched against the version name, causes that version
to be ignored. For example, 'beta' could be specified to ignore version names
that include that text.
+@node Update Check Name
+@section Update Check Name
+
+@cindex Update Check Name
+
+When checking for updates (via @code{Update Check Mode}) this can be used to
+specify the package name to search for. Useful when apps have a static package
+name but change it programmatically in some app flavors, by e.g. appending
+".open" or ".free" at the end of the package name.
+
+You can also use @code{Ignore} to ignore package name searching. This should
+only be used in some specific cases, for example if the app's build.gradle
+file does not contain the package name.
+
@node Update Check Data
@section Update Check Data
This field is normally automatically updated - see Update Check Mode.
+If not set or set to @code{0}, clients will recommend the highest version they
+can, as if the @code{Current Version Code} was infinite.
+
This is converted to (@code{<marketvercode>}) in the public index file.
@node No Source Since
@enumerate
@item
-Use custom ant build steps to execute virtually anything as the user doing
+Use custom Ant build steps to execute virtually anything as the user doing
the build.
@item
Access the keystore.
@section Setting up a build server
In addition to the basic setup previously described, you will also need
-a Vagrant-compatible Debian Testing base box called 'testing32' (or testing64
+a Vagrant-compatible Debian Testing base box called 'jessie32' (or jessie64
for a 64-bit VM, if you want it to be much slower, and require more disk
space).
for example, having subversion v1.6 on the host and v1.7 on the guest
would fail.
-Unless you're very trusting. you should create one of these for yourself
-from verified standard Debian installation media. However, you could skip
-over the next few paragraphs (and sacrifice some security) by downloading
-@url{https://f-droid.org/testing32.box}.
+@subsection Creating the Debian base box
+
+The output of this step is a minimal Debian VM that has support for remote
+login and provisioning.
+
+Unless you're very trusting, you should create one of these for yourself
+from verified standard Debian installation media. However, by popular
+demand, the @code{makebuildserver} script will automatically download a
+prebuilt image unless instructed otherwise. If you choose to use the
+prebuilt image, you may safely skip the rest of this section.
Documentation for creating a base box can be found at
@url{http://docs.vagrantup.com/v1/docs/base_boxes.html}.
-1 in @code{/etc/grub/default} and then run @code{update-grub}.
@end enumerate
+@subsection Creating the F-Droid base box
-With this base box available, you should then create @code{makebs.config.py},
-using @code{makebs.config.sample.py} as a reference - look at the settings and
+The next step in the process is to create @code{makebs.config.py},
+using @code{./examples/makebs.config.py} as a reference - look at the settings and
documentation there to decide if any need changing to suit your environment.
There is a path for retrieving the base box if it doesn't exist, and an apt
proxy definition, both of which may need customising for your environment.
running the android tools. For example, if you have
@code{buildserver/addons/cache/platforms/android-19.tar.gz} that will be
used when installing the android-19 platform, instead of re-downloading it
-using @code{android update sdk --no-ui -t android-19}.
+using @code{android update sdk --no-ui -t android-19}. It is possible to
+create the cache files of this additions from a local installation of the
+SDK including these:
+
+@example
+cd /path/to/android-sdk/platforms
+tar czf android-19.tar.gz android-19
+mv android-19.tar.gz /path/to/buildserver/addons/cache/platforms/
+@end example
+
+If you have already built a buildserver it is also possible to get this
+files directly from the buildserver:
+
+@example
+vagrant ssh -- -C 'tar -C ~/android-sdk/platforms czf android-19.tar.gz android-19'
+vagrant ssh -- -C 'cat ~/android-sdk/platforms/android-19.tar.gz' > /path/to/fdroidserver/buildserver/cache/platforms/android19.tar.gz
+@end example
Once it's complete you'll have a new base box called 'buildserver' which is
what's used for the actual builds. You can then build packages as normal,