1 \input texinfo @c -*-texinfo-*-
3 @setfilename fdroid.info
4 @documentencoding UTF-8
5 @settitle F-Droid Server Manual
9 This manual is for the F-Droid repository server tools.
11 Copyright @copyright{} 2010, 2011, 2012, 2013, 2014, 2015 Ciaran Gultnieks
13 Copyright @copyright{} 2011 Henrik Tunedal, Michael Haas, John Sullivan
15 Copyright @copyright{} 2013 David Black
17 Copyright @copyright{} 2013, 2014 Daniel MartÃ
20 Permission is granted to copy, distribute and/or modify this document
21 under the terms of the GNU Free Documentation License, Version 1.3
22 or any later version published by the Free Software Foundation;
23 with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
24 A copy of the license is included in the section entitled "GNU
25 Free Documentation License".
30 @title F-Droid Server Manual
31 @author Ciaran Gultnieks and the F-Droid project
33 @vskip 0pt plus 1filll
49 * System Requirements::
51 * Simple Binary Repository::
52 * Building Applications::
53 * Importing Applications::
58 * GNU Free Documentation License::
65 The F-Droid server tools provide various scripts and tools that are used
66 to maintain the main F-Droid application repository. You can use these same
67 tools to create your own additional or alternative repository for publishing,
68 or to assist in creating, testing and submitting metadata to the main
72 @node System Requirements
73 @chapter System Requirements
77 The system requirements for using the tools will vary depending on your
78 intended usage. At the very least, you'll need:
85 To be sure of being able to process all apk files without error, you need
86 2.7.7 or later. See @code{http://bugs.python.org/issue14315}.
88 The Android SDK Tools and Build-tools.
89 Note that F-Droid does not assume that you have the Android SDK in your
90 @code{PATH}: these directories will be specified in your repository
91 configuration. Recent revisions of the SDK have @code{aapt} located in
92 android-sdk/build-tools/ and it may be necessary to make a symlink to it in
93 android-sdk/platform-tools/
96 If you intend to build applications from source you'll also need most, if not
97 all, of the following:
101 JDK (Debian package openjdk-6-jdk): openjdk-6 is recommended though openjdk-7
104 VCS clients: svn, git, git-svn, hg, bzr
106 A keystore for holding release keys. (Safe, secure and well backed up!)
109 If you intend to use the 'Build Server' system, for secure and clean builds
110 (highly recommended), you will also need:
114 VirtualBox (debian package virtualbox)
116 Ruby (debian packages ruby and rubygems)
118 Vagrant (unpackaged, tested on v1.4.3)
120 Paramiko (debian package python-paramiko)
122 Imaging (debian package python-imaging)
124 Magic (debian package python-magic)
127 On the other hand, if you want to build the apps directly on your system
128 without the 'Build Server' system, you may need:
132 All SDK platforms requested by the apps you want to build
133 (The Android SDK is made available by Google under a proprietary license but
134 within that, the SDK platforms, support library and some other components are
135 under the Apache license and source code is provided.
136 Google APIs, used for building apps using Google Maps, are free to the extent
137 that the library comes pre-installed on the device.
138 Google Play Services, Google Admob and others are proprietary and shouldn't be
139 included in the main F-Droid repository.)
141 A version of the Android NDK
143 Ant with Contrib Tasks (Debian packages ant and ant-contrib)
145 Maven (Debian package maven)
147 JavaCC (Debian package javacc)
149 Miscellaneous packages listed in
150 buildserver/cookbooks/fdroidbuild-general/recipes/default.rb
151 of the F-Droid server repository
157 @cindex setup, installation
159 Because the tools and data will always change rapidly, you will almost
160 certainly want to work from a git clone of the tools at this stage. To
164 git clone https://gitlab.com/fdroid/fdroidserver.git
167 You now have lots of stuff in the fdroidserver directory, but the most
168 important is the 'fdroid' command script which you run to perform all tasks.
169 This script is always run from a repository data directory, so the
170 most sensible thing to do next is to put your new fdroidserver directory
175 To do anything, you'll need at least one repository data directory. It's
176 from this directory that you run the @code{fdroid} command to perform all
177 repository management tasks. You can either create a brand new one, or
178 grab a copy of the data used by the main F-Droid repository:
181 git clone https://gitlab.com/fdroid/fdroiddata.git
184 Regardless of the intended usage of the tools, you will always need to set
185 up some basic configuration details. This is done by creating a file called
186 @code{config.py} in the data directory. You should do this by copying the
187 example file (@code{./examples/config.py}) from the fdroidserver project to
188 your data directory and then editing according to the instructions within.
190 Once configured in this way, all the functionality of the tools is accessed
191 by running the @code{fdroid} command. Run it on its own to get a list of the
192 available sub-commands.
194 You can follow any command with @code{--help} to get a list of additional
195 options available for that command.
202 @node Simple Binary Repository
203 @chapter Simple Binary Repository
207 If you want to maintain a simple repository hosting only binary APKs obtained
208 and compiled elsewhere, the process is quite simple:
212 Set up the server tools, as described in Setup.
214 Make a directory for your repository. This is the directory from which you
215 will do all the work with your repository. Create a config file there, called
216 @code{config.py}, by copying @code{./examples/config.py} from the server
217 project and editing it.
219 Within that, make a directory called @code{repo} and put APK files in it.
221 Run @code{fdroid update}.
223 If it reports that any metadata files are missing, you can create them
224 in the @code{metadata} directory and run it again.
226 To ease creation of metadata files, run @code{fdroid update} with the @code{-c}
227 option. It will create 'skeleton' metadata files that are missing, and you can
228 then just edit them and fill in the details.
230 Then, if you've changed things, run @code{fdroid update} again.
232 Running @code{fdroid update} adds an Icons directory into the repo directory,
233 and also creates the repository index (index.xml, and also index.jar if you've
234 configured the system to use a signed index).
236 Publish the resulting contents of the @code{repo} directory to your web server.
239 Following the above process will result in a @code{repo} directory, which you
240 simply need to push to any HTTP (or preferably HTTPS) server to make it
243 While some information about the applications (and versions thereof) is
244 retrieved directly from the APK files, most comes from the corresponding file
245 in the @code{metadata} directory. The metadata file covering ALL versions of a
246 particular application is named @code{package.id.txt} where package.id is the
247 unique identifier for that package.
249 See the Metadata chapter for details of what goes in the metadata file. All
250 fields are relevant for binary APKs, EXCEPT for @code{Build:} entries, which
254 @node Building Applications
255 @chapter Building Applications
257 Instead of (or as well as) including binary APKs from external sources in a
258 repository, you can build them directly from the source code.
260 Using this method, it is is possible to verify that the application builds
261 correctly, corresponds to the source code, and contains only free software.
262 Unforunately, in the Android world, it seems to be very common for an
263 application supplied as a binary APK to present itself as Free Software
264 when in fact some or all of the following are true:
268 The source code (either for a particular version, or even all versions!) is
269 unavailable or incomplete.
271 The source code is not capable of producing the actual binary supplied.
273 The 'source code' contains binary files of unknown origin, or with proprietary
277 For this reason, source-built applications are the preferred method for the
278 main F-Droid repository, although occasionally for technical or historical
279 reasons, exceptions are made to this policy.
281 When building applications from source, it should be noted that you will be
282 signing them (all APK files must be signed to be installable on Android) with
283 your own key. When an application is already installed on a device, it is not
284 possible to upgrade it in place to a new version signed with a different key
285 without first uninstalling the original. This may present an inconvenience to
286 users, as the process of uninstalling loses any data associated with the
287 previous installation.
289 The process for managing a repository for built-from-source applications is
290 very similar to that described in the Simple Binary Repository chapter,
291 except now you need to:
295 Include Build entries in the metadata files.
297 Run @code{fdroid build} to build any applications that are not already built.
299 Run @code{fdroid publish} to finalise packaging and sign any APKs that have
304 @section More about "fdroid build"
306 When run without any parameters, @code{fdroid build} will build any and all
307 versions of applications that you don't already have in the @code{repo}
308 directory (or more accurately, the @code{unsigned} directory). There are various
309 other things you can do. As with all the tools, the @code{--help} option is
310 your friend, but a few annotated examples and discussion of the more common
313 To build a single version of a single application, you could run the
317 ./fdroid build org.fdroid.fdroid:16
320 This attempts to build version code 16 (which is version 0.25) of the F-Droid
321 client. Many of the tools recognise arguments as packages, allowing their
322 activity to be limited to just a limited set of packages.
324 If the build above was successful, two files will have been placed in the
325 @code{unsigned} directory:
328 org.fdroid.fdroid_16.apk
329 org.fdroid.fdroid_16_src.tar.gz
332 The first is the (unsigned) APK. You could sign this with a debug key and push
333 it direct to your device or an emulator for testing. The second is a source
334 tarball containing exactly the source that was used to generate the binary.
336 If you were intending to publish these files, you could then run:
342 The source tarball would move to the @code{repo} directory (which is the
343 directory you would push to your web server). A signed and zip-aligned version
344 of the APK would also appear there, and both files would be removed from the
345 @code{unsigned} directory.
347 If you're building purely for the purposes of testing, and not intending to
348 push the results to a repository, at least yet, the @code{--test} option can be
349 used to direct output to the @code{tmp} directory instead of @code{unsigned}.
350 A similar effect could by achieved by simply deleting the output files from
351 @code{unsigned} after the build, but with the risk of forgetting to do so!
353 Along similar lines (and only in conjunction with @code{--test}, you can use
354 @code{--force} to force a build of a Disabled application, where normally it
355 would be completely ignored. Similarly a version that was found to contain
356 ELFs or known non-free libraries can be forced to build. See also —
357 @code{scanignore=} and @code{scandelete=} in the @code{Build:} section.
359 If the build was unsuccessful, you can find out why by looking at the output
360 in the logs/ directory. If that isn't illuminating, try building the app the
361 regular way, step by step: android update project, ndk-build, ant debug.
363 Note that source code repositories often contain prebuilt libraries. If the
364 app is being considered for the main F-Droid repository, it is important that
365 all such prebuilts are built either via the metadata or by a reputable third
369 @section Direct Installation
371 You can also build and install directly to a connected device or emulator
372 using the @code{fdroid install} command. If you do this without passing
373 packages as arguments then all the latest built and signed version available
374 of each package will be installed . In most cases, this will not be what you
375 want to do, so execution will stop straight away. However, you can override
376 this if you're sure that's what you want, by using @code{--all}. Note that
377 currently, no sanity checks are performed with this mode, so if the files in
378 the signed output directory were modified, you won't be notified.
381 @node Importing Applications
382 @chapter Importing Applications
384 To help with starting work on including a new application, @code{fdroid import}
385 will take a URL and optionally some other parameters, and attempt to construct
386 as much information as possible by analysing the source code. Basic usage is:
389 ./fdroid import --url=http://address.of.project
392 For this to work, the URL must point to a project format that the script
393 understands. Currently this is limited to one of the following:
397 Gitorious - @code{https://gitorious.org/PROJECTNAME/REPONAME}
399 Github - @code{https://github.com/USER/PROJECT}
401 Google Code - @code{http://code.google.com/p/PROJECT/}
402 Supports git, svn and hg repos.
404 Some Google Code projects have multiple repositories, identified by a
405 dropdown list on the @code{source/checkout} page. To access one other than
406 the default, specify its name using the @code{--repo} switch.
408 Bitbucket - @code{https://bitbucket.org/USER/PROJECT/}
410 Git - @code{git://REPO}
413 Depending on the project type, more or less information may be gathered. For
414 example, the license will be retrieved from a Google Code project, but not a
415 GitHub one. A bare repo url, such as the git:// one, is the least preferable
416 optional of all, since you will have to enter much more information manually.
418 If the import is successful, a metadata file will be created. You will need to
419 edit this further to check the information, and fill in the blanks.
421 If it fails, you'll be told why. If it got as far as retrieving the source
422 code, you can inspect it further by looking in @code{tmp/importer} where a full
425 A frequent cause of initial failure is that the project directory is actually
426 a subdirectory in the repository. In this case, run the importer again using
427 the @code{--subdir} option to tell it where. It will not attempt to determine
428 this automatically, since there may be several options.
436 Information used by update.py to compile the public index comes from two
441 the APK files in the repo directory, and
443 the metadata files in the metadata directory.
446 The metadata files are simple, easy to edit text files, always named as the
447 application's package ID with '.txt' appended.
449 Note that although the metadata files are designed to be easily read and
450 writable by humans, they are also processed and written by various scripts.
451 They are capable of rewriting the entire file when necessary. Even so,
452 the structure and comments will be preserved correctly, although the order
453 of fields will be standardised. (In the event that the original file was
454 in a different order, comments are considered as being attached to the field
455 following them). In fact, you can standardise all the metadata in a single
456 command, without changing the functional content, by running:
459 fdroid rewritemetadata
462 The following sections describe the fields recognised within the file.
488 * Update Check Mode::
489 * Update Check Ignore::
490 * Vercode Operation::
491 * Update Check Name::
492 * Update Check Data::
495 * Current Version Code::
502 Any number of categories for the application to be placed in. There is no
503 fixed list of categories - both the client and the web site will automatically
504 show any categories that exist in any applications. However, if your metadata
505 is intended for the main F-Droid repository, you should use one of the
506 existing categories (look at the site/client), or discuss the proposal to add
509 Categories must be separated by a single comma character, ','. For backwards
510 compatibility, F-Droid will use the first category given as <category> element
511 for older clients to at least see one category.
513 This is converted to (@code{<categories>}) in the public index file.
520 The overall license for the application, or in certain cases, for the
533 GNU GPL version 2 or later
541 GNU GPL version 3 or later
545 An unspecified GPL version. Use this only as a last resort or if there is
546 some confusion over compatiblity of component licenses: particularly the use of
547 Apache libraries with GPLv2 source code.
551 Afferro GPL version 3.
563 BSD license - the original '4-clause' version.
567 BSD license - the new, or modified, version.
571 This is converted to (@code{<license>}) in the public index file.
578 The name of the application as can best be retrieved from the source code.
579 This is done so that the commitupdates script can put a familiar name in the
580 description of commits created when a new update of the application is
581 found. The Auto Name entry is generated automatically when @code{fdroid
582 checkupdates} is run.
589 The name of the application. Normally, this field should not be present since
590 the application's correct name is retrieved from the APK file. However, in a
591 situation where an APK contains a bad or missing application name, it can be
592 overridden using this. Note that this only overrides the name in the list of
593 apps presented in the client; it doesn't changed the name or application label
601 Comma-separated list of application IDs that this app provides. In other
602 words, if the user has any of these apps installed, F-Droid will show this app
603 as installed instead. It will also appear if the user clicks on urls linking
604 to the other app IDs. Useful when an app switches package name, or when you
605 want an app to act as multiple apps.
612 The URL for the application's web site. If there is no relevant web site, this
613 can be omitted (or left blank).
615 This is converted to (@code{<web>}) in the public index file.
622 The URL to view or obtain the application's source code. This should be
623 something human-friendly. Machine-readable source-code is covered in the
626 This is converted to (@code{<source>}) in the public index file.
629 @section Issue Tracker
631 @cindex Issue Tracker
633 The URL for the application's issue tracker. Optional, since not all
634 applications have one.
636 This is converted to (@code{<tracker>}) in the public index file.
643 The URL for the application's changelog. Optional, since not all
644 applications have one.
646 This is converted to (@code{<changelog>}) in the public index file.
653 The URL to donate to the project. This should be the project's donate page
656 It is possible to use a direct PayPal link here, if that is all that is
657 available. However, bear in mind that the developer may not be aware of
658 that direct link, and if they later changed to a different PayPal account,
659 or the PayPal link format changed, things could go wrong. It is always
660 best to use a link that the developer explicitly makes public, rather than
661 something that is auto-generated 'button code'.
663 This is converted to (@code{<donate>}) in the public index file.
670 The project's Flattr (http://flattr.com) ID, if it has one. This should be
671 a numeric ID, such that (for example) https://flattr.com/thing/xxxx leads
672 directly to the page to donate to the project.
674 This is converted to (@code{<flattr>}) in the public index file.
681 A bitcoin address for donating to the project.
683 This is converted to (@code{<bitcoin>}) in the public index file.
690 A litecoin address for donating to the project.
697 A brief summary of what the application is. Since the summary is only allowed
698 one line on the list of the F-Droid client, keeping it to within 50 characters
699 will ensure it fits most screens.
706 A full description of the application, relevant to the latest version.
707 This can span multiple lines (which should be kept to a maximum of 80
708 characters), and is terminated by a line containing a single '.'.
710 Basic MediaWiki-style formatting can be used. Leaving a blank line starts a
711 new paragraph. Surrounding text with @code{''} make it italic, and with
712 @code{'''} makes it bold.
714 You can link to another app in the repo by using @code{[[app.id]]}. The link
715 will be made appropriately whether in the Android client, the web repo
716 browser or the wiki. The link text will be the apps name.
718 Links to web addresses can be done using @code{[http://example.com Text]}.
720 For both of the above link formats, the entire link (from opening to closing
721 square bracket) must be on the same line.
723 Bulletted lists are done by simply starting each item with a @code{*} on
724 a new line, and numbered lists are the same but using @code{#}. There is
725 currently no support for nesting lists - you can have one level only.
727 It can be helpful to note information pertaining to updating from an
728 earlier version; whether the app contains any prebuilts built by the
729 upstream developers or whether non-free elements were removed; whether the
730 app is in rapid development or whether the latest version lags behind the
731 current version; whether the app supports multiple architectures or whether
732 there is a maximum SDK specified (such info not being recorded in the index).
734 This is converted to (@code{<desc>}) in the public index file.
736 @node Maintainer Notes
737 @section Maintainer Notes
739 @cindex Maintainer Notes
741 This is a multi-line field using the same rules and syntax as the description.
742 It's used to record notes for F-Droid maintainers to assist in maintaining and
743 updating the application in the repository.
745 This information is also published to the wiki.
752 The type of repository - for automatic building from source. If this is not
753 specified, automatic building is disabled for this application. Possible
775 The repository location. Usually a git: or svn: URL, for example.
777 The git-svn option connects to an SVN repository, and you specify the URL in
778 exactly the same way, but git is used as a back-end. This is preferable for
779 performance reasons, and also because a local copy of the entire history is
780 available in case the upstream repository disappears. (It happens!). In
781 order to use Tags as update check mode for this VCS type, the URL must have
782 the tags= special argument set. Likewise, if you intend to use the
783 RepoManifest/branch scheme, you would want to specify branches= as well.
784 Finally, trunk= can also be added. All these special arguments will be passed
785 to "git svn" in order, and their values must be relative paths to the svn repo
787 Here's an example of a complex git-svn Repo URL:
788 http://svn.code.sf.net/p/project/code/svn;trunk=trunk;tags=tags;branches=branches
790 If the Repo Type is @code{srclib}, then you must specify the name of the
791 according srclib .txt file. For example if @code{scrlibs/FooBar.txt} exist
792 and you want to use this srclib, then you have to set Repo to
800 Any number of these fields can be present, each specifying a version to
801 automatically build from source. The value is a comma-separated list.
806 The above specifies to build version 1.2, which has a version code of 12.
807 The @code{commit=} parameter specifies the tag, commit or revision number from
808 which to build it in the source repository. It is the only mandatory flag,
809 which in this case could for example be @code{commit=v1.2}.
811 In addition to the three, always required, parameters described above,
812 further parameters can be added (in name=value format) to apply further
813 configuration to the build. These are (roughly in order of application):
817 @item disable=<message>
818 Disables this build, giving a reason why. (For backwards compatibility, this
819 can also be achieved by starting the commit ID with '!')
821 The purpose of this feature is to allow non-buildable releases (e.g. the source
822 is not published) to be flagged, so the scripts don't generate repeated
823 messages about them. (And also to record the information for review later).
824 If an apk has already been built, disabling causes it to be deleted once
825 @code{fdroid update} is run; this is the procedure if ever a version has to
829 Specifies to build from a subdirectory of the checked out source code.
830 Normally this directory is changed to before building,
833 Use if the project (git only) has submodules - causes @code{git submodule
834 update --init --recursive} to be executed after the source is cloned.
835 Submodules are reset and cleaned like the main app repository itself before
839 As for 'prebuild', but runs on the source code BEFORE any other processing
842 You can use $$SDK$$, $$NDK$$ and $$MVN3$$ to substitute the paths to the
843 android SDK and NDK directories, and maven 3 executable respectively. The
844 following per-build variables are available likewise: $$VERSION$$,
845 $$VERCODE$$ and $$COMMIT$$.
848 The sdk location in the repo is in an old format, or the build.xml is
849 expecting such. The 'new' format is sdk.dir while the VERY OLD format
850 is sdk-location. Typically, if you get a message along the lines of:
851 "com.android.ant.SetupTask cannot be found" when trying to build, then
852 try enabling this option.
854 @item target=<target>
855 Specifies a particular SDK target for compilation, overriding the value
856 defined in the code by upstream. This has different effects depending on what
857 build system used — this flag currently affects Ant, Maven and Gradle projects
858 only. Note that this does not change the target SDK in the
859 AndroidManifest.xml, which determines the level of features that can be
860 included in the build.
862 In the case of an Ant project, it modifies project.properties of the app and
863 possibly sub-projects. This is likely to cause the whole build.xml to be
864 rewritten, which is fine if it's a 'standard' android file or doesn't already
865 exist, but not a good idea if it's heavily customised.
867 @item update=<auto/dirs>
868 By default, 'android update' is used in Ant builds to generate or update the
869 project and all its referenced projects. Specifying update=no bypasses that.
870 Note that this is useless in builds that don't use Ant.
872 Default value is '@code{auto}', which recursively uses the paths in
873 project.properties to find all the subprojects to update.
875 Otherwise, the value can be a comma-separated list of directories in which to
876 run 'android update' relative to the application directory.
879 Adds a java.encoding property to local.properties with the given
880 value. Generally the value will be 'utf-8'. This is picked up by the
881 SDK's ant rules, and forces the Java compiler to interpret source
882 files with this encoding. If you receive warnings during the compile
883 about character encodings, you probably need this.
885 @item forceversion=yes
886 If specified, the package version in AndroidManifest.xml is replaced
887 with the version name for the build as specified in the metadata.
889 This is useful for cases when upstream repo failed to update it for
890 specific tag; to build an arbitrary revision; to make it apparent that
891 the version differs significantly from upstream; or to make it apparent
892 which architecture or platform the apk is designed to run on.
894 @item forcevercode=yes
895 If specified, the package version code in the AndroidManifest.xml is
896 replaced with the version code for the build. See also forceversion.
898 @item rm=<path1>[,<path2>,...]
899 Specifies the relative paths of files or directories to delete before
900 the build is done. The paths are relative to the base of the build
901 directory - i.e. the root of the directory structure checked out from
902 the source respository - not necessarily the directory that contains
905 Multiple files/directories can be specified by separating them with ','.
906 Directories will be recursively deleted.
908 @item extlibs=<lib1>[,<lib2>,...]
909 Comma-separated list of external libraries (jar files) from the
910 @code{build/extlib} library, which will be placed in the @code{libs} directory
913 @item srclibs=[n:]a@@r,[n:]b@@r1,...
914 Comma-separated list of source libraries or Android projects. Each item is of
915 the form name@@rev where name is the predefined source library name and rev is
916 the revision or tag to use in the respective source control.
918 For Ant projects, you can optionally append a number with a colon at the
919 beginning of a srclib item to automatically place it in project.properties as
920 a library under the specified number. For example, if you specify
921 @code{1:somelib@@1.0}, f-droid will automatically do the equivalent of the
922 legacy practice @code{prebuild=echo "android.library.reference.1=$$somelib$$"
923 >> project.properties}.
925 Each srclib has a metadata file under srclibs/ in the repository directory,
926 and the source code is stored in build/srclib/.
927 Repo Type: and Repo: are specified in the same way as for apps; Subdir: can be
928 a comma separated list, for when directories are renamed by upstream; Update
929 Project: updates the projects in the working directory and one level down;
930 Prepare: can be used for any kind of preparation: in particular if you need to
931 update the project with a particular target. You can then also use $$name$$ in
932 the init/prebuild/build command to substitute the relative path to the library
933 directory, but it could need tweaking if you've changed into another directory.
936 Apply patch(es). 'x' names one (or more - comma-seperated) files within a
937 directory below the metadata, with the same name as the metadata file but
938 without the extension. Each of these patches is applied to the code in turn.
941 Specifies a shell command (or commands - chain with &&) to run before the
942 build takes place. Backslash can be used as an escape character to insert
943 literal commas, or as the last character on a line to join that line with the
944 next. It has no special meaning in other contexts; in particular, literal
945 backslashes should not be escaped.
947 The command runs using bash.
949 Note that nothing should be built during this prebuild phase - scanning of the
950 code and building of the source tarball, for example, take place after this.
951 For custom actions that actually build things or produce binaries, use 'build'
954 You can use $$name$$ to substitute the path to a referenced srclib - see
955 the @code{srclib} directory for details of this.
957 You can use $$SDK$$, $$NDK$$ and $$MVN3$$ to substitute the paths to the
958 android SDK and NDK directories, and Maven 3 executable respectively e.g.
959 for when you need to run @code{android update project} explicitly. The
960 following per-build variables are available likewise: $$VERSION$$, $$VERCODE$$
963 @item scanignore=<path1>[,<path2>,...]
964 Enables one or more files/paths to be excluded from the scan process.
965 This should only be used where there is a very good reason, and
966 probably accompanied by a comment explaining why it is necessary.
968 When scanning the source tree for problems, matching files whose relative
969 paths start with any of the paths given here are ignored.
971 @item scandelete=<path1>[,<path2>,...]
972 Similar to scanignore=, but instead of ignoring files under the given paths,
973 it tells f-droid to delete the matching files directly.
976 As for 'prebuild', but runs during the actual build phase (but before the
977 main Ant/Maven build). Use this only for actions that do actual building.
978 Any prepartion of the source code should be done using 'init' or 'prebuild'.
980 Any building that takes place before build= will be ignored, as either Ant,
981 mvn or gradle will be executed to clean the build environment right before
982 build= (or the final build) is run.
984 You can use $$SDK$$, $$NDK$$ and $$MVN3$$ to substitute the paths to the
985 android SDK and NDK directories, and maven 3 executable respectively. The
986 following per-build variables are available likewise: $$VERSION$$,
987 $$VERCODE$$ and $$COMMIT$$.
989 @item buildjni=[yes|no|<dir list>]
990 Enables building of native code via the ndk-build script before doing
991 the main Ant build. The value may be a list of directories relative
992 to the main application directory in which to run ndk-build, or 'yes'
993 which corresponds to '.' . Using explicit list may be useful to build
994 multi-component projects.
996 The build and scan processes will complain (refuse to build) if this
997 parameter is not defined, but there is a @code{jni} directory present.
998 If the native code is being built by other means like a Gradle task, you
999 can specify @code{no} here to avoid that. However, if the native code is
1000 actually not required or used, remove the directory instead (using
1001 @code{rm=jni} for example). Using @code{buildjni=no} when the jni code
1002 isn't used nor built will result in an error saying that native
1003 libraries were expected in the resulting package.
1006 Version of the NDK to use in this build. Defaults to the latest NDK release
1007 that included legacy toolchains, so as to not break builds that require
1008 toolchains no longer included in current versions of the NDK.
1010 The buildserver supports r9b with its legacy toolchains and the latest release
1011 as of writing this document, r10e. You may add support for more versions by
1012 adding them to 'ndk_paths' in your config file.
1014 @item gradle=<flavour1>[,<flavour2>,...]
1015 Build with Gradle instead of Ant, specifying what flavours to use. Flavours
1016 are case sensitive since the path to the output apk is as well.
1018 If only one flavour is given and it is 'yes' or 'main', no flavour will be
1019 used. Note that for projects with flavours, you must specify at least one
1020 valid flavour since 'yes' or 'main' will build all of them separately.
1022 @item maven=yes[@@<dir>]
1023 Build with Maven instead of Ant. An extra @@<dir> tells f-droid to run Maven
1024 inside that relative subdirectory. Sometimes it is needed to use @@.. so that
1025 builds happen correctly.
1027 @item preassemble=<task1>[,<task2>,...]
1028 List of Gradle tasks to be run before the assemble task in a Gradle project
1031 @item antcommands=<target1>[,<target2>,...]
1032 Specify an alternate set of Ant commands (target) instead of the default
1033 'release'. It can't be given any flags, such as the path to a build.xml.
1035 @item output=path/to/output.apk
1036 To be used when app is built with a tool other than the ones natively
1037 supported, like GNU Make. The given path will be where the build= set of
1038 commands should produce the final unsigned release apk.
1041 Don't check that the version name and code in the resulting apk are
1042 correct by looking at the build output - assume the metadata is
1043 correct. This takes away a useful level of sanity checking, and should
1044 only be used if the values can't be extracted.
1048 Another example, using extra parameters:
1050 @samp{Build Version:1.09.03,10903,45,subdir=Timeriffic,oldsdkloc=yes}
1053 @section AntiFeatures
1055 @cindex AntiFeatures
1057 This is optional - if present, it contains a comma-separated list of any of
1058 the following values, describing an anti-feature the application has.
1059 It is a good idea to mention the reasons for the anti-feature(s) in the
1065 @samp{Ads} - the application contains advertising.
1068 @samp{Tracking} - the application tracks and reports your activity to
1069 somewhere without your consent. It's commonly used for when developers
1070 obtain crash logs without the user's consent, or when an app is useless
1071 without some kind of authentication.
1074 @samp{NonFreeNet} - the application relies on computational services that
1075 are impossible to replace or that the replacement cannot be connected to
1076 without major changes to the app.
1079 @samp{NonFreeAdd} - the application promotes non-free add-ons, such that the
1080 app is effectively an advert for other non-free software and such software is
1081 not clearly labelled as such.
1084 @samp{NonFreeDep} - the application depends on a non-free application (e.g.
1085 Google Maps) - i.e. it requires it to be installed on the device, but does not
1089 @samp{UpstreamNonFree} - the application is or depends on non-free software.
1090 This does not mean that non-free software is included with the app: Most
1091 likely, it has been patched in some way to remove the non-free code. However,
1092 functionality may be missing.
1101 If this field is present, the application does not get put into the public
1102 index. This allows metadata to be retained while an application is temporarily
1103 disabled from being published. The value should be a description of why the
1104 application is disabled. No apks or source code archives are deleted: to purge
1105 an apk see the Build Version section or delete manually for developer builds.
1106 The field is therefore used when an app has outlived it's usefulness, because
1107 the source tarball is retained.
1110 @section Requires Root
1112 @cindex Requires Root
1114 Set this optional field to "Yes" if the application requires root
1115 privileges to be usable. This lets the client filter it out if the
1116 user so desires. Whether root is required or not, it is good to give
1117 a paragraph in the description to the conditions on which root may be
1118 asked for and the reason for it.
1120 @node Archive Policy
1121 @section Archive Policy
1123 @cindex Archive Policy
1125 This determines the policy for moving old versions of an app to the archive
1126 repo, if one is configured. The configuration sets a default maximum number
1127 of versions kept in the main repo, after which older ones are moved to the
1128 archive. This app-specific policy setting can override that.
1130 Currently the only supported format is "n versions", where n is the number
1131 of versions to keep.
1133 @node Update Check Mode
1134 @section Update Check Mode
1136 @cindex Update Check Mode
1138 This determines the method using for determining when new releases are
1139 available - in other words, the updating of the Current Version and Current
1140 Version Code fields in the metadata by the @code{fdroid checkupdates} process.
1146 @code{None} - No checking is done because there's no appropriate automated way
1147 of doing so. Updates should be checked for manually. Use this, for example,
1148 when deploying betas or patched versions; when builds are done in a directory
1149 different to where the AndroidManifest.xml is; if the developers use the
1150 Gradle build system and store version info in a separate file; if the
1151 developers make a new branch for each release and don't make tags; or if you've
1152 changed the package name or version code logic.
1154 @code{Static} - No checking is done - either development has ceased or new versions
1155 are not desired. This method is also used when there is no other checking method
1156 available and the upstream developer keeps us posted on new versions.
1158 @code{RepoManifest} - At the most recent commit, the AndroidManifest.xml file
1159 is looked for in the directory where it was found in the the most recent build.
1160 The appropriateness of this method depends on the development process used by
1161 the application's developers. You should not specify this method unless you're
1162 sure it's appropriate. For example, some developers bump the version when
1163 commencing development instead of when publishing.
1164 It will return an error if the AndroidManifest.xml has moved to a different
1165 directory or if the package name has changed.
1166 The current version that it gives may not be accurate, since not all
1167 versions are fit to be published. Therefore, before building, it is often
1168 necessary to check if the current version has been published somewhere by the
1169 upstream developers, either by checking for apks that they distribute or for
1170 tags in the source code repository.
1172 It currently works for every repository type to different extents, except
1173 the srclib repo type. For git, git-svn and hg repo types, you may use
1174 "RepoManifest/yourbranch" as UCM so that "yourbranch" would be the branch used
1175 in place of the default one. The default values are "master" for git,
1176 "default" for hg and none for git-svn (it stays in the same branch).
1177 On the other hand, branch support hasn't been implemented yet in bzr and svn,
1178 but RepoManifest may still be used without it.
1180 @code{RepoTrunk} - For svn and git-svn repositories, especially those who
1181 don't have a bundled AndroidManifest.xml file, the Tags and RepoManifest
1182 checks will not work, since there is no version information to obtain. But,
1183 for those apps who automate their build process with the commit ref that HEAD
1184 points to, RepoTrunk will set the Current Version and Current Version Code to
1187 @code{Tags} - The AndroidManifest.xml file in all tagged revisions in the
1188 source repository is checked, looking for the highest version code. The
1189 appropriateness of this method depends on the development process used by the
1190 application's developers. You should not specify this method unless you're sure
1191 it's appropriate. It shouldn't be used if the developers like to tag betas or
1192 are known to forget to tag releases. Like RepoManifest, it will not return the
1193 correct value if the directory containing the AndroidManifest.xml has moved.
1194 Despite these caveats, it is the often the favourite update check mode.
1196 It currently only works for git, hg, bzr and git-svn repositories. In the case
1197 of the latter, the repo URL must contain the path to the trunk and tags or
1198 else no tags will be found.
1200 Optionally append a regex pattern at the end - separated with a space - to
1201 only check the tags matching said pattern. Useful when apps tag non-release
1202 versions such as X.X-alpha, so you can filter them out with something like
1203 @code{.*[0-9]$} which requires tag names to end with a digit.
1205 @code{HTTP} - HTTP requests are used to determine the current version code and
1206 version name. This is controlled by the @code{Update Check Data} field, which
1207 is of the form @code{urlcode|excode|urlver|exver}.
1209 Firstly, if @code{urlcode} is non-empty, the document from that URL is
1210 retrieved, and matched against the regular expression @code{excode}, with the
1211 first group becoming the version code.
1213 Secondly, if @code{urlver} is non-empty, the document from that URL is
1214 retrieved, and matched against the regular expression @code{exver}, with the
1215 first group becoming the version name. The @code{urlver} field can be set to
1216 simply '.' which says to use the same document returned for the version code
1217 again, rather than retrieving a different one.
1220 @node Vercode Operation
1221 @section Vercode Operation
1223 @cindex Vercode Operation
1225 Operation to be applied to the vercode obtained by the defined @code{Update
1226 Check Mode}. @code{%c} will be replaced by the actual vercode, and the whole
1227 string will be passed to python's @code{eval} function.
1229 Especially useful with apps that we want to compile for different ABIs, but
1230 whose vercodes don't always have trailing zeros. For example, with
1231 @code{Vercode Operation} set at something like @code{%c*10 + 4}, we will be
1232 able to track updates and build up to four different versions of every
1235 @node Update Check Ignore
1236 @section Update Check Ignore
1238 @cindex Update Check Ignore
1240 When checking for updates (via @code{Update Check Mode}) this can be used to
1241 specify a regex which, if matched against the version name, causes that version
1242 to be ignored. For example, 'beta' could be specified to ignore version names
1243 that include that text.
1245 @node Update Check Name
1246 @section Update Check Name
1248 @cindex Update Check Name
1250 When checking for updates (via @code{Update Check Mode}) this can be used to
1251 specify the package name to search for. Useful when apps have a static package
1252 name but change it programmatically in some app flavors, by e.g. appending
1253 ".open" or ".free" at the end of the package name.
1255 @node Update Check Data
1256 @section Update Check Data
1258 @cindex Update Check Data
1260 Used in conjunction with @code{Update Check Mode} for certain modes.
1262 @node Auto Update Mode
1263 @section Auto Update Mode
1265 @cindex Auto Update Mode
1267 This determines the method using for auto-generating new builds when new
1268 releases are available - in other words, adding a new Build Version line to the
1270 This happens in conjunction with the 'Update Check Mode' functionality - i.e.
1271 when an update is detected by that, it is also processed by this.
1277 @code{None} - No auto-updating is done
1279 @code{Version} - Identifies the target commit (i.e. tag) for the new build based
1280 on the given version specification, which is simply text in which %v and %c are
1281 replaced with the required version name and version code respectively.
1283 For example, if an app always has a tag "2.7.2" corresponding to version 2.7.2,
1284 you would simply specify "Version %v". If an app always has a tag "ver_1234"
1285 for a version with version code 1234, you would specify "Version ver_%c".
1287 Additionally, a suffix can be added to the version name at this stage, to
1288 differentiate F-Droid's build from the original. Continuing the first example
1289 above, you would specify that as "Version +-fdroid %v" - "-fdroid" is the suffix.
1293 @node Current Version
1294 @section Current Version
1296 @cindex Current Version
1298 The name of the version that is current. There may be newer versions of the
1299 application than this (e.g. betas), and there will almost certainly be older
1300 ones. This should be the one that is recommended for general use.
1301 In the event that there is no source code for the current version, or that
1302 non-free libraries are being used, this would ideally be the latest
1303 version that is still free, though it may still be expedient to
1304 retain the automatic update check — see No Source Since.
1306 This field is normally automatically updated - see Update Check Mode.
1308 This is converted to (@code{<marketversion>}) in the public index file.
1310 @node Current Version Code
1311 @section Current Version Code
1313 @cindex Current Version Code
1315 The version code corresponding to the Current Version field. Both these fields
1316 must be correct and matching although it's the current version code that's
1317 used by Android to determine version order and by F-Droid client to determine
1318 which version should be recommended.
1320 This field is normally automatically updated - see Update Check Mode.
1322 This is converted to (@code{<marketvercode>}) in the public index file.
1324 @node No Source Since
1325 @section No Source Since
1327 @cindex No Source Since
1329 In case we are missing the source code for the Current Version reported by
1330 Upstream, or that non-free elements have been introduced, this defines the
1331 first version that began to miss source code.
1332 Apps that are missing source code for just one or a few versions, but provide
1333 source code for newer ones are not to be considered here - this field is
1334 intended to illustrate which apps do not currently distribute source code, and
1335 since when have they been doing so.
1337 @node Update Processing
1338 @chapter Update Processing
1342 There are various mechanisms in place for automatically detecting that updates
1343 are available for applications, with the @code{Update Check Mode} field in the
1344 metadata determining which method is used for a particular application.
1346 Running the @code{fdroid checkupdates} command will apply this method to each
1347 application in the repository and update the @code{Current Version} and
1348 @code{Current Version Code} fields in the metadata accordingly.
1350 As usual, the @code{-p} option can be used with this, to restrict processing
1351 to a particular application.
1353 Note that this only updates the metadata such that we know what the current
1354 published/recommended version is. It doesn't make that version available in
1355 the repository - for that, see the next section.
1359 Adding updates (i.e. new versions of applications already included in the
1360 repository) happens in two ways. The simple case is applications where the
1361 APK files are binaries, retrieved from a developer's published build. In this
1362 case, all that's required is to place the new binary in the @code{Repo}
1363 directory, and the next run of @code{fdroid update} will pick it up.
1365 For applications built from source, it is necessary to add a new
1366 @code{Build Version} line to the metadata file. At the very least, the version
1367 name, version code and commit will be different. It is also possible that the
1368 additional build flags will change between versions.
1370 For processing multiple updates in the metadata at once, it can be useful to
1371 run @code{fdroid update --interactive}. This will check all the applications
1372 in the repository, and where updates are required you will be prompted to
1373 [E]dit the metadata, [I]gnore the update, or [Q]uit altogether.
1376 @chapter Build Server
1378 The Build Server system isolates the builds for each package within a clean,
1379 isolated and secure throwaway virtual machine environment.
1383 Building applications in this manner on a large scale, especially with the
1384 involvement of automated and/or unattended processes, could be considered
1385 a dangerous pastime from a security perspective. This is even more the case
1386 when the products of the build are also distributed widely and in a
1387 semi-automated ("you have updates available") fashion.
1389 Assume that an upstream source repository is compromised. A small selection
1390 of things that an attacker could do in such a situation:
1394 Use custom Ant build steps to execute virtually anything as the user doing
1397 Access the keystore.
1399 Modify the built apk files or source tarballs for other applications in the
1402 Modify the metadata (which includes build scripts, which again, also includes
1403 the ability to execute anything) for other applications in the repository.
1406 Through complete isolation, the repurcussions are at least limited to the
1407 application in question. Not only is the build environment fresh for each
1408 build, and thrown away afterwards, but it is also isolated from the signing
1411 Aside from security issues, there are some applications which have strange
1412 requirements such as custom versions of the NDK. It would be impractical (or
1413 at least extremely messy) to start modifying and restoring the SDK on a
1414 multi-purpose system, but within the confines of a throwaway single-use
1415 virtual machine, anything is possible.
1417 All this is in addition to the obvious advantage of having a standardised
1418 and completely reproducible environment in which builds are made. Additionally,
1419 it allows for specialised custom build environments for particular
1422 @section Setting up a build server
1424 In addition to the basic setup previously described, you will also need
1425 a Vagrant-compatible Debian Testing base box called 'testing32' (or testing64
1426 for a 64-bit VM, if you want it to be much slower, and require more disk
1429 You can use a different version or distro for the base box, so long as you
1430 don't expect any help making it work. One thing to be aware of is that
1431 working copies of source trees are moved from the host to the guest, so
1432 for example, having subversion v1.6 on the host and v1.7 on the guest
1435 @subsection Creating the Debian base box
1437 The output of this step is a minimal Debian VM that has support for remote
1438 login and provisioning.
1440 Unless you're very trusting, you should create one of these for yourself
1441 from verified standard Debian installation media. However, by popular
1442 demand, the @code{makebuildserver} script will automatically download a
1443 prebuilt image unless instructed otherwise. If you choose to use the
1444 prebuilt image, you may safely skip the rest of this section.
1446 Documentation for creating a base box can be found at
1447 @url{http://docs.vagrantup.com/v1/docs/base_boxes.html}.
1449 In addition to carefully following the steps described there, you should
1450 consider the following:
1454 It is advisable to disable udev network device persistence, otherwise any
1455 movement of the VM between machines, or reconfiguration, will result in
1458 For a Debian/Ubuntu default install, just
1459 @code{touch /etc/udev/rules.d/75-persistent-net-generator.rules} to turn
1460 off rule generation, and at the same time, get rid of any rules it's
1461 already created in @code{/etc/udev/rules.d/70-persistent-net.rules}.
1463 Unless you want the VM to become totally inaccessible following a failed
1464 boot, you need to set @code{GRUB_RECORDFAIL_TIMEOUT} to a value other than
1465 -1 in @code{/etc/grub/default} and then run @code{update-grub}.
1468 @subsection Creating the F-Droid base box
1470 The next step in the process is to create @code{makebs.config.py},
1471 using @code{./examples/makebs.config.py} as a reference - look at the settings and
1472 documentation there to decide if any need changing to suit your environment.
1473 There is a path for retrieving the base box if it doesn't exist, and an apt
1474 proxy definition, both of which may need customising for your environment.
1475 You can then go to the @code{fdroidserver} directory and run this:
1481 This will take a long time, and use a lot of bandwidth - most of it spent
1482 installing the necessary parts of the Android SDK for all the various
1483 platforms. Luckily you only need to do it occasionally. Once you have a
1484 working build server image, if the recipes change (e.g. when packages need
1485 to be added) you can just run that script again and the existing one will
1486 be updated in place.
1488 The main sdk/ndk downloads will automatically be cached to speed things
1489 up the next time, but there's no easy way of doing this for the longer
1490 sections which use the SDK's @code{android} tool to install platforms,
1491 add-ons and tools. However, instead of allowing automatic caching, you
1492 can supply a pre-populated cache directory which includes not only these
1493 downloads, but also .tar.gz files for all the relevant additions. If the
1494 provisioning scripts detect these, they will be used in preference to
1495 running the android tools. For example, if you have
1496 @code{buildserver/addons/cache/platforms/android-19.tar.gz} that will be
1497 used when installing the android-19 platform, instead of re-downloading it
1498 using @code{android update sdk --no-ui -t android-19}. It is possible to
1499 create the cache files of this additions from a local installation of the
1500 SDK including these:
1503 cd /path/to/android-sdk/platforms
1504 tar czf android-19.tar.gz android-19
1505 mv android-19.tar.gz /path/to/buildserver/addons/cache/platforms/}
1508 If you have already built a buildserver it is also possible to get this
1509 files directly from the buildserver:
1512 vagrant ssh -- -C 'tar -C ~/android-sdk/platforms czf android-19.tar.gz android-19'
1513 vagrant ssh -- -C 'cat ~/android-sdk/platforms/android-19.tar.gz' > /path/to/fdroidserver/buildserver/cache/platforms/android19.tar.gz
1516 Once it's complete you'll have a new base box called 'buildserver' which is
1517 what's used for the actual builds. You can then build packages as normal,
1518 but with the addition of the @code{--server} flag to @code{fdroid build} to
1519 instruct it to do all the hard work within the virtual machine.
1521 The first time a build is done, a new virtual machine is created using the
1522 'buildserver' box as a base. A snapshot of this clean machine state is saved
1523 for use in future builds, to improve performance. You can force discarding
1524 of this snapshot and rebuilding from scratch using the @code{--resetserver}
1525 switch with @code{fdroid build}.
1530 There are two kinds of signing involved in running a repository - the signing
1531 of the APK files generated from source builds, and the signing of the repo
1532 index itself. The latter is optional, but very strongly recommended.
1534 @section Repo Index Signing
1536 When setting up the repository, one of the first steps should be to generate
1537 a signing key for the repository index. This will also create a keystore, which
1538 is a file that can be used to hold this and all other keys used. Consider the
1539 location, security and backup status of this file carefully, then create it as
1542 @code{keytool -genkey -v -keystore my.keystore -alias repokey -keyalg RSA -keysize 2048 -validity 10000}
1544 In the above, replace 'my.keystore' with the name of the keystore file to be
1545 created, and 'repokey' with a name to identify the repo index key by.
1547 You'll be asked for a password for the keystore, AND a password for the key.
1548 They shouldn't be the same. In between, you'll be asked for some identifying
1549 details which will go in the certificate.
1551 The two passwords entered go into @code{config.py}, as @code{keystorepass} and
1552 @code{keypass} respectively. The path to the keystore file, and the alias you
1553 chose for the key also go into that file, as @code{keystore} and
1554 @code{repo_keyalias} respectively.
1556 @section Package Signing
1558 With the repo index signing configured, all that remains to be done for package
1559 signing to work is to set the @code{keydname} field in @code{config.py} to
1560 contain the same identifying details you entered before.
1562 A new key will be generated using these details, for each application that is
1563 built. (If a specific key is required for a particular application, this system
1564 can be overridden using the @code{keyaliases} config settings.
1567 @node GNU Free Documentation License
1568 @appendix GNU Free Documentation License