1 \input texinfo @c -*-texinfo-*-
3 @setfilename fdroid.info
4 @documentencoding UTF-8
5 @settitle F-Droid Server Manual
9 This manual is for the F-Droid repository server tools.
11 Copyright @copyright{} 2010, 2011, 2012, 2013 Ciaran Gultnieks
13 Copyright @copyright{} 2011 Henrik Tunedal, Michael Haas, John Sullivan
15 Copyright @copyright{} 2013 David Black, Daniel Martí
18 Permission is granted to copy, distribute and/or modify this document
19 under the terms of the GNU Free Documentation License, Version 1.3
20 or any later version published by the Free Software Foundation;
21 with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
22 A copy of the license is included in the section entitled "GNU
23 Free Documentation License".
28 @title F-Droid Server Manual
29 @author Ciaran Gultnieks and the F-Droid project
31 @vskip 0pt plus 1filll
47 * System Requirements::
49 * Simple Binary Repository::
50 * Building Applications::
51 * Importing Applications::
56 * GNU Free Documentation License::
63 The F-Droid server tools provide various scripts and tools that are used
64 to maintain the main F-Droid application repository. You can use these same
65 tools to create your own additional or alternative repository for publishing,
66 or to assist in creating, testing and submitting metadata to the main
70 @node System Requirements
71 @chapter System Requirements
75 The system requirements for using the tools will vary depending on your
76 intended usage. At the very least, you'll need:
84 The Android SDK Tools and Build-tools.
85 Note that F-Droid does not assume that you have the Android SDK in your
86 @code{PATH}: these directories will be specified in your repository
87 configuration. Recent revisions of the SDK have @code{aapt} located in
88 android-sdk/build-tools/ and it may be necessary to make a symlink to it in
89 android-sdk/platform-tools/
92 If you intend to build applications from source you'll also need most, if not
93 all, of the following:
97 JDK (Debian package openjdk-6-jdk): openjdk-6 is recommended though openjdk-7
100 VCS clients: svn, git, git-svn, hg, bzr
102 A keystore for holding release keys. (Safe, secure and well backed up!)
105 If you intend to use the 'Build Server' system, for secure and clean builds
106 (highly recommended), you will also need:
110 VirtualBox (debian package virtualbox)
112 Ruby (debian packages ruby and rubygems)
114 Vagrant (unpackaged) Be sure to use 1.3.x because 1.4.x is completely broken
115 (at the time of writing, the forthcoming 1.4.3 might work)
117 Paramiko (debian package python-paramiko)
119 Imaging (debian package python-imaging)
121 Magic (debian package python-magic)
124 On the other hand, if you want to build the apps directly on your system
125 without the 'Build Server' system, you may need:
129 All SDK platforms requested by the apps you want to build
130 (The Android SDK is made available by Google under a proprietary license but
131 within that, the SDK platforms, support library and some other components are
132 under the Apache license and source code is provided.
133 Google APIs, used for building apps using Google Maps, are free to the extent
134 that the library comes pre-installed on the device.
135 Google Play Services, Google Admob and others are proprietary and shouldn't be
136 included in the main F-Droid repository.)
138 A version of the Android NDK
140 Ant with Contrib Tasks (Debian packages ant and ant-contrib)
142 Maven (Debian package maven)
144 JavaCC (Debian package javacc)
146 Miscellaneous packages listed in
147 buildserver/cookbooks/fdroidbuild-general/recipes/default.rb
148 of the F-Droid server repository
154 @cindex setup, installation
156 Because the tools and data will always change rapidly, you will almost
157 certainly want to work from a git clone of the tools at this stage. To
161 git clone git://gitorious.org/f-droid/fdroidserver.git
164 You now have lots of stuff in the fdroidserver directory, but the most
165 important is the 'fdroid' command script which you run to perform all tasks.
166 This script is always run from a repository data directory, so the
167 most sensible thing to do next is to put your new fdroidserver directory
172 To do anything, you'll need at least one repository data directory. It's
173 from this directory that you run the @code{fdroid} command to perform all
174 repository management tasks. You can either create a brand new one, or
175 grab a copy of the data used by the main F-Droid repository:
178 git clone git://gitorious.org/f-droid/fdroiddata.git
181 Regardless of the intended usage of the tools, you will always need to set
182 up some basic configuration details. This is done by creating a file called
183 @code{config.py} in the data directory. You should do this by copying the
184 example file (@code{config.sample.py}) from the fdroidserver project to your
185 data directory and then editing according to the instructions within.
187 Once configured in this way, all the functionality of the tools is accessed
188 by running the @code{fdroid} command. Run it on its own to get a list of the
189 available sub-commands.
191 You can follow any command with @code{--help} to get a list of additional
192 options available for that command.
199 @node Simple Binary Repository
200 @chapter Simple Binary Repository
204 If you want to maintain a simple repository hosting only binary APKs obtained
205 and compiled elsewhere, the process is quite simple:
209 Set up the server tools, as described in Setup.
211 Make a directory for your repository. This is the directory from which you
212 will do all the work with your repository. Create a config file there, called
213 @code{config.py}, by copying the @code{config.sample.py} from the server
214 project and editing it.
216 Within that, make a directory called @code{repo} and put APK files in it.
218 Run @code{fdroid update}.
220 If it reports that any metadata files are missing, you can create them
221 in the @code{metadata} directory and run it again.
223 To ease creation of metadata files, run @code{fdroid update} with the @code{-c}
224 option. It will create 'skeleton' metadata files that are missing, and you can
225 then just edit them and fill in the details.
227 Then, if you've changed things, run @code{fdroid update} again.
229 Running @code{fdroid update} adds an Icons directory into the repo directory,
230 and also creates the repository index (index.xml, and also index.jar if you've
231 configured the system to use a signed index).
233 Publish the resulting contents of the @code{repo} directory to your web server.
236 Following the above process will result in a @code{repo} directory, which you
237 simply need to push to any HTTP (or preferably HTTPS) server to make it
240 While some information about the applications (and versions thereof) is
241 retrieved directly from the APK files, most comes from the corresponding file
242 in the @code{metadata} directory. The metadata file covering ALL versions of a
243 particular application is named @code{package.id.txt} where package.id is the
244 unique identifier for that package.
246 See the Metadata chapter for details of what goes in the metadata file. All
247 fields are relevant for binary APKs, EXCEPT for 'Build Version' entries, which
251 @node Building Applications
252 @chapter Building Applications
254 Instead of (or as well as) including binary APKs from external sources in a
255 repository, you can build them directly from the source code.
257 Using this method, it is is possible to verify that the application builds
258 correctly, corresponds to the source code, and contains only free software.
259 Unforunately, in the Android world, it seems to be very common for an
260 application supplied as a binary APK to present itself as Free Software
261 when in fact some or all of the following are true:
265 The source code (either for a particular version, or even all versions!) is
266 unavailable or incomplete.
268 The source code is not capable of producing the actual binary supplied.
270 The 'source code' contains binary files of unknown origin, or with proprietary
274 For this reason, source-built applications are the preferred method for the
275 main F-Droid repository, although occasionally for technical or historical
276 reasons, exceptions are made to this policy.
278 When building applications from source, it should be noted that you will be
279 signing them (all APK files must be signed to be installable on Android) with
280 your own key. When an application is already installed on a device, it is not
281 possible to upgrade it in place to a new version signed with a different key
282 without first uninstalling the original. This may present an inconvenience to
283 users, as the process of uninstalling loses any data associated with the
284 previous installation.
286 The process for managing a repository for built-from-source applications is
287 very similar to that described in the Simple Binary Repository chapter,
288 except now you need to:
292 Include Build Version entries in the metadata files.
294 Run @code{fdroid build} to build any applications that are not already built.
296 Run @code{fdroid publish} to finalise packaging and sign any APKs that have
301 @section More about "fdroid build"
303 When run without any parameters, @code{fdroid build} will build any and all
304 versions of applications that you don't already have in the @code{repo}
305 directory (or more accurately, the @code{unsigned} directory). There are various
306 other things you can do. As with all the tools, the @code{--help} option is
307 your friend, but a few annotated examples and discussion of the more common
310 To build a single version of a single application, you could run the
314 ./fdroid build org.fdroid.fdroid:16
317 This attempts to build version code 16 (which is version 0.25) of the F-Droid
318 client. Many of the tools recognise arguments as packages, allowing their
319 activity to be limited to just a limited set of packages.
321 If the build above was successful, two files will have been placed in the
322 @code{unsigned} directory:
325 org.fdroid.fdroid_16.apk
326 org.fdroid.fdroid_16_src.tar.gz
329 The first is the (unsigned) APK. You could sign this with a debug key and push
330 it direct to your device or an emulator for testing. The second is a source
331 tarball containing exactly the source that was used to generate the binary.
333 If you were intending to publish these files, you could then run:
339 The source tarball would move to the @code{repo} directory (which is the
340 directory you would push to your web server). A signed and zip-aligned version
341 of the APK would also appear there, and both files would be removed from the
342 @code{unsigned} directory.
344 If you're building purely for the purposes of testing, and not intending to
345 push the results to a repository, at least yet, the @code{--test} option can be
346 used to direct output to the @code{tmp} directory instead of @code{unsigned}.
347 A similar effect could by achieved by simply deleting the output files from
348 @code{unsigned} after the build, but with the risk of forgetting to do so!
350 Along similar lines (and only in conjunction with @code{--test}, you can use
351 @code{--force} to force a build of a Disabled application, where normally it
352 would be completely ignored. Similarly a version that was found to contain
353 ELFs or known non-free libraries can be forced to build. See also —
354 scanignore= and scandelete= in the Build Version section.
356 If the build was unsuccessful, you can find out why by looking at the output
357 in the logs/ directory. If that isn't illuminating, try building the app the
358 regular way, step by step: android update project, ndk-build, ant debug.
360 Note that source code repositories often contain prebuilt libraries. If the
361 app is being considered for the main F-Droid repository, it is important that
362 all such prebuilts are built either via the metadata or by a reputable third
366 @section Direct Installation
368 You can also build and install directly to a connected device or emulator
369 using the @code{fdroid install} command. If you do this without passing
370 packages as arguments then all the latest built and signed version available
371 of each package will be installed . In most cases, this will not be what you
372 want to do, so execution will stop straight away. However, you can override
373 this if you're sure that's what you want, by using @code{--all}. Note that
374 currently, no sanity checks are performed with this mode, so if the files in
375 the signed output directory were modified, you won't be notified.
378 @node Importing Applications
379 @chapter Importing Applications
381 To help with starting work on including a new application, @code{fdroid import}
382 will take a URL and optionally some other parameters, and attempt to construct
383 as much information as possible by analysing the source code. Basic usage is:
386 ./fdroid import --url=http://address.of.project
389 For this to work, the URL must point to a project format that the script
390 understands. Currently this is limited to one of the following:
394 Gitorious - @code{https://gitorious.org/PROJECTNAME/REPONAME}
396 Github - @code{https://github.com/USER/PROJECT}
398 Google Code - @code{http://code.google.com/p/PROJECT/}
399 Supports git, svn and hg repos.
401 Some Google Code projects have multiple repositories, identified by a
402 dropdown list on the @code{source/checkout} page. To access one other than
403 the default, specify its name using the @code{--repo} switch.
405 Bitbucket - @code{https://bitbucket.org/USER/PROJECT/}
407 Git - @code{git://REPO}
410 Depending on the project type, more or less information may be gathered. For
411 example, the license will be retrieved from a Google Code project, but not a
412 GitHub one. A bare repo url, such as the git:// one, is the least preferable
413 optional of all, since you will have to enter much more information manually.
415 If the import is successful, a metadata file will be created. You will need to
416 edit this further to check the information, and fill in the blanks.
418 If it fails, you'll be told why. If it got as far as retrieving the source
419 code, you can inspect it further by looking in @code{tmp/importer} where a full
422 A frequent cause of initial failure is that the project directory is actually
423 a subdirectory in the repository. In this case, run the importer again using
424 the @code{--subdir} option to tell it where. It will not attempt to determine
425 this automatically, since there may be several options.
433 Information used by update.py to compile the public index comes from two
438 the APK files in the repo directory, and
440 the metadata files in the metadata directory.
443 The metadata files are simple, easy to edit text files, always named as the
444 application's package ID with '.txt' appended.
446 Note that although the metadata files are designed to be easily read and
447 writable by humans, they are also processed and written by various scripts.
448 They are capable of rewriting the entire file when necessary. Even so,
449 the structure and comments will be preserved correctly, although the order
450 of fields will be standardised. (In the event that the original file was
451 in a different order, comments are considered as being attached to the field
452 following them). In fact, you can standardise all the metadata in a single
453 command, without changing the functional content, by running:
456 fdroid rewritemetadata
459 The following sections describe the fields recognised within the file.
484 * Update Check Mode::
485 * Vercode Operation::
486 * Update Check Data::
489 * Current Version Code::
496 Any number of categories for the application to be placed in. There is no
497 fixed list of categories - both the client and the web site will automatically
498 show any categories that exist in any applications. However, if your metadata
499 is intended for the main F-Droid repository, you should use one of the
500 existing categories (look at the site/client), or discuss the proposal to add
503 Categories must be separated by a single comma character, ','. For backwards
504 compatibility, F-Droid will use the first category given as <category> element
505 for older clients to at least see one category.
507 This is converted to (@code{<categories>}) in the public index file.
514 The overall license for the application, or in certain cases, for the
527 GNU GPL version 2 or later
535 GNU GPL version 3 or later
539 An unspecified GPL version. Use this only as a last resort or if there is
540 some confusion over compatiblity of component licenses: particularly the use of
541 Apache libraries with GPLv2 source code.
545 Afferro GPL version 3.
557 BSD license - the original '4-clause' version.
561 BSD license - the new, or modified, version.
565 This is converted to (@code{<license>}) in the public index file.
572 The name of the application as can best be retrieved from the source code.
573 This is done so that the commitupdates script can put a familiar name in the
574 description of commits created when a new update of the application is
575 found. The Auto Name entry is generated automatically when @code{fdroid
576 checkupdates} is run.
583 The name of the application. Normally, this field should not be present since
584 the application's correct name is retrieved from the APK file. However, in a
585 situation where an APK contains a bad or missing application name, it can be
586 overridden using this. Note that this only overrides the name in the list of
587 apps presented in the client; it doesn't changed the name or application label
595 Comma-separated list of application IDs that this app provides. In other
596 words, if the user has any of these apps installed, F-Droid will show this app
597 as installed instead. It will also appear if the user clicks on urls linking
598 to the other app IDs. Useful when an app switches package name, or when you
599 want an app to act as multiple apps.
606 The URL for the application's web site.
608 This is converted to (@code{<web>}) in the public index file.
615 The URL to view or obtain the application's source code. This should be
616 something human-friendly. Machine-readable source-code is covered in the
619 This is converted to (@code{<source>}) in the public index file.
622 @section Issue Tracker
624 @cindex Issue Tracker
626 The URL for the application's issue tracker. Optional, since not all
627 applications have one.
629 This is converted to (@code{<tracker>}) in the public index file.
636 The URL to donate to the project. This should be the project's donate page
639 It is possible to use a direct PayPal link here, if that is all that is
640 available. However, bear in mind that the developer may not be aware of
641 that direct link, and if they later changed to a different PayPal account,
642 or the PayPal link format changed, things could go wrong. It is always
643 best to use a link that the developer explicitly makes public, rather than
644 something that is auto-generated 'button code'.
646 This is converted to (@code{<donate>}) in the public index file.
653 The project's Flattr (http://flattr.com) ID, if it has one. This should be
654 a numeric ID, such that (for example) https://flattr.com/thing/xxxx leads
655 directly to the page to donate to the project.
657 This is converted to (@code{<flattr>}) in the public index file.
664 A bitcoin address for donating to the project.
666 This is converted to (@code{<bitcoin>}) in the public index file.
673 A litecoin address for donating to the project.
680 A brief summary of what the application is. Since the summary is only allowed
681 one line on the list of the F-Droid client, keeping it to within 32 characters
682 will ensure it fits even on the smallest screens.
689 A full description of the application, relevant to the latest version.
690 This can span multiple lines (which should be kept to a maximum of 80
691 characters), and is terminated by a line containing a single '.'.
693 Basic MediaWiki-style formatting can be used. Leaving a blank line starts a
694 new paragraph. Surrounding text with @code{''} make it italic, and with
695 @code{'''} makes it bold.
697 You can link to another app in the repo by using @code{[[app.id]]}. The link
698 will be made appropriately whether in the Android client, the web repo
699 browser or the wiki. The link text will be the apps name.
701 Links to web addresses can be done using @code{[http://example.com Text]}.
703 For both of the above link formats, the entire link (from opening to closing
704 square bracket) must be on the same line.
706 Bulletted lists are done by simply starting each item with a @code{*} on
707 a new line, and numbered lists are the same but using @code{#}. There is
708 currently no support for nesting lists - you can have one level only.
710 It can be helpful to note information pertaining to updating from an
711 earlier version; whether the app contains any prebuilts built by the
712 upstream developers or whether non-free elements were removed; whether the
713 app is in rapid development or whether the latest version lags behind the
714 current version; whether the app supports multiple architectures or whether
715 there is a maximum SDK specified (such info not being recorded in the index).
717 This is converted to (@code{<desc>}) in the public index file.
719 @node Maintainer Notes
720 @section Maintainer Notes
722 @cindex Maintainer Notes
724 This is a multi-line field using the same rules and syntax as the description.
725 It's used to record notes for F-Droid maintainers to assist in maintaining and
726 updating the application in the repository.
728 This information is also published to the wiki.
735 The type of repository - for automatic building from source. If this is not
736 specified, automatic building is disabled for this application. Possible
758 The repository location. Usually a git: or svn: URL, for example.
760 The git-svn option connects to an SVN repository, and you specify the URL in
761 exactly the same way, but git is used as a back-end. This is preferable for
762 performance reasons, and also because a local copy of the entire history is
763 available in case the upstream repository disappears. (It happens!). In
764 order to use Tags as update check mode for this VCS type, the URL must have
765 the tags= special argument set. Likewise, if you intend to use the
766 RepoManifest/branch scheme, you would want to specify branches= as well.
767 Finally, trunk= can also be added. All these special arguments will be passed
768 to "git svn" in order, and their values must be relative paths to the svn repo
770 Here's an example of a complex git-svn Repo URL:
771 http://svn.code.sf.net/p/project/code/svn;trunk=trunk;tags=tags;branches=branches
773 For a Subversion repo that requires authentication, you can precede the repo
774 URL with username:password@ and those parameters will be passed as @option{--username}
775 and @option{--password} to the SVN checkout command. (This now works for both
778 If the Repo Type is @code{srclib}, then you must specify the name of the
779 according srclib .txt file. For example if @code{scrlibs/FooBar.txt} exist
780 and you want to use this srclib, then you have to set Repo to
784 @section Build Version
786 @cindex Build Version
788 Any number of these fields can be present, each specifying a version to
789 automatically build from source. The value is a comma-separated list.
792 @samp{Build Version:0.12,3,651696a49be2cd7db5ce6a2fa8185e31f9a20035}
794 The above specifies to build version 0.12, which has a version code of 3.
795 The third parameter specifies the tag, commit or revision number from
796 which to build it in the source repository.
798 In addition to the three, always required, parameters described above,
799 further parameters can be added (in name=value format) to apply further
800 configuration to the build. These are (roughly in order of application):
804 @item disable=<message>
805 Disables this build, giving a reason why. (For backwards compatibility, this
806 can also be achieved by starting the commit ID with '!')
808 The purpose of this feature is to allow non-buildable releases (e.g. the source
809 is not published) to be flagged, so the scripts don't generate repeated
810 messages about them. (And also to record the information for review later).
811 If an apk has already been built, disabling causes it to be deleted once
812 @code{fdroid update} is run; this is the procedure if ever a version has to
816 Specifies to build from a subdirectory of the checked out source code.
817 Normally this directory is changed to before building,
820 Use if the project (git only) has submodules - causes git submodule
821 init and update to be executed after the source is cloned.
824 As for 'prebuild', but runs on the source code BEFORE any other processing
827 You can use $$SDK$$, $$NDK$$ and $$MVN3$$ to substitute the paths to the
828 android SDK and NDK directories, and maven 3 executable respectively.
831 The sdk location in the repo is in an old format, or the build.xml is
832 expecting such. The 'new' format is sdk.dir while the VERY OLD format
833 is sdk-location. Typically, if you get a message along the lines of:
834 "com.android.ant.SetupTask cannot be found" when trying to build, then
835 try enabling this option.
837 @item target=<target>
838 Specifies a particular SDK target for compilation, overriding the
839 project.properties of the app and possibly sub-projects. Note that this does
840 not change the target SDK in the AndroidManifest.xml — the level of features
841 that can be included in the build. This is likely to cause the whole build.xml
842 to be rewritten, which is fine if it's a 'standard' android file or doesn't
843 already exist, but not a good idea if it's heavily customised. If you get an
844 error about invalid target, first try @code{init=rm -rf bin/}; otherwise this
845 parameter should do the trick.
847 Please note that gradle builds should be using compilesdk=.
849 @item compilesdk=<level>
850 Practically accomplishes the same that target= does when used in ant and maven
851 projects. compilesdk= is used rather than target= so as to not cause any more
852 confusion. It only takes effect on gradle builds in the build.gradle file,
853 thus using it in any other case is not wise.
856 By default, 'android update project' is used to generate or update the
857 project and all its referenced projects. Specifying update=no bypasses that.
859 Specifiying update=force forces rebuilding of the build.xml file at the
860 same time - this is frequently needed with r14 of the Android platform
861 tools. Be aware of any customisations in build.xml when using
864 Default value is '@code{auto}', which uses the paths used in the
865 project.properties file to find out what project paths to update.
867 Otherwise, value can be a semicolon-separated list of directories in
868 which to run 'android update project' relative to the main
869 application directory (which may include '@code{subdir}' parameter).
872 Adds a java.encoding property to local.properties with the given
873 value. Generally the value will be 'utf-8'. This is picked up by the
874 SDK's ant rules, and forces the Java compiler to interpret source
875 files with this encoding. If you receive warnings during the compile
876 about character encodings, you probably need this.
878 @item forceversion=yes
879 If specified, the package version in AndroidManifest.xml is replaced
880 with the version name for the build as specified in the metadata.
882 This is useful for cases when upstream repo failed to update it for
883 specific tag; to build an arbitrary revision; to make it apparent that
884 the version differs significantly from upstream; or to make it apparent
885 which architecture or platform the apk is designed to run on.
887 @item forcevercode=yes
888 If specified, the package version code in the AndroidManifest.xml is
889 replaced with the version code for the build. See also forceversion.
891 @item rm=<relpath1;relpath2;...>
892 Specifies the relative paths of files or directories to delete before
893 the build is done. The paths are relative to the base of the build
894 directory - i.e. the root of the directory structure checked out from
895 the source respository - not necessarily the directory that contains
898 Multiple files/directories can be specified by separating them with ';'.
899 Directories will be recursively deleted.
902 Modifies any instances of string resources that use multiple
903 formatting arguments, but don't use positional notation. For example,
904 "Hello %s, %d" becomes "Hello %1$s, %2$d". Newer versions of the
905 Android platform tools enforce this sensible standard. If you get
906 error messages relating to that, you need to enable this.
909 Like fixtrans, but deals with an even older issue relating to
910 'unescaped apostrophes' in translation strings.
913 Specifies a list of external libraries (jar files) from the
914 @code{build/extlib} library, which will be placed in the @code{libs} directory
915 of the project. Separate items with semicolons.
917 @item srclibs=a@@r;b@@r1;
918 Specifies a list of source libraries or Android projects. Separate items with
919 semicolons, and each item is of the form name@@rev where name is the predefined
920 source library name and rev is the revision or tag in source control to use.
922 Each srclib has a metadata file under srclibs/ in the repository directory,
923 and the source code is stored in build/srclib/.
924 Repo Type: and Repo: are specified in the same way as for apps; Subdir: can be
925 a comma separated list, for when directories are renamed by upstream; Update
926 Project: updates the projects in the working directory and one level down;
927 Prepare: can be used for any kind of preparation: in particular if you need to
928 update the project with a particular target. You can then also use $$name$$ in
929 the init/prebuild/build command to substitute the relative path to the library
930 directory, but it could need tweaking if you've changed into another directory.
933 Apply patch(es). 'x' names one (or more - comma-seperated)
934 files within a directory below the metadata, with the same
935 name as the metadata file but without the extension. Each of
936 these patches is applied to the code in turn.
939 Specifies a shell command (or commands - chain with &&) to run before
940 the build takes place. Backslash can be used as an escape character to
941 insert literal commas, or as the last character on a line to join that
942 line with the next. It has no special meaning in other contexts; in
943 particular, literal backslashes should not be escaped.
945 The command runs using bash.
947 Note that nothing should be build during this prebuild phase - scanning
948 of the code and building of the source tarball, for example, take place
949 after this. For custom actions that actually build things, use 'build'
952 You can use $$name$$ to substitute the path to a referenced srclib - see
953 the @code{srclib} directory for details of this.
955 You can use $$SDK$$, $$NDK$$ and $$MVN3$$ to substitute the paths to the
956 android SDK and NDK directories, and maven 3 executable respectively e.g.
957 for when you need to run @code{android update project} explicitly.
959 @item scanignore=path1;path2;...
960 Enables one or more files/paths to be exlcuded from the scan process.
961 This should only be used where there is a very good reason, and
962 probably accompanied by a comment explaining why it is necessary.
964 When scanning the source tree for problems, matching files whose relative
965 paths start with any of the paths given here are ignored.
967 @item scandelete=path1;path2;...
968 Similar to scanignore=, but instead of ignoring files under the given paths,
969 it tells f-droid to delete the matching files directly.
972 As for 'prebuild', but runs during the actual build phase (but before the
973 main ant/maven build). Use this only for actions that do actual building.
974 Any prepartion of the source code should be done using 'init' or 'prebuild'.
976 Any building that takes place before build= will be ignored, as either ant,
977 mvn or gradle will be executed to clean the build environment right before
978 build= (or the final build) is run.
980 You can use $$SDK$$, $$NDK$$ and $$MVN3$$ to substitute the paths to the
981 android SDK and NDK directories, and maven 3 executable respectively.
983 @item buildjni=[yes|no|<dir list>]
984 Enables building of native code via the ndk-build script before doing
985 the main ant build. The value may be a list of directories relative
986 to the main application directory in which to run ndk-build, or 'yes'
987 which corresponds to '.' . Using explicit list may be useful to build
988 multi-component projects.
990 The build and scan processes will complain (refuse to build) if this
991 parameter is not defined, but there is a @code{jni} directory present.
992 If the native code is being built by other means, you can specify
993 @code{no} here to avoid that. However, if the native code is actually
994 not required, remove the directory instead (using @code{prebuild} for
997 @item gradle=<flavour>[@@<dir>]
998 Build with gradle instead of ant, specifying what flavour to assemble.
999 If <flavour> is 'yes', 'main' or empty, no flavour will be used. Note
1000 that this will not work on projects with flavours, since it will build
1001 all flavours and there will be no 'main' build.
1002 If @@<dir> is attached to <flavour>, then the gradle tasks will be run in that
1003 directory. This might be necessary if gradle needs to be run in the parent
1004 directory, in which case one would use 'gradle=<flavour>@..'.
1006 @item maven=yes[@@<dir>]
1007 Build with maven instead of ant. Like gradle, an extra @@<dir> tells f-droid
1008 to run maven inside that relative subdirectory.
1010 @item preassemble=<task1> <task2>
1011 Space-separated list of gradle tasks to be run before the assemble task
1012 in a gradle project build.
1015 Normally the build output (apk) is expected to be in the bin
1016 subdirectory below the ant build files. If the project is configured
1017 to put it elsewhere, that can be specified here, relative to the base
1018 of the checked out repo. Not yet implemented for gradle.
1020 @item antcommand=xxx
1021 Specify an alternate ant command (target) instead of the default
1022 'release'. It can't be given any flags, such as the path to a build.xml.
1025 Don't check that the version name and code in the resulting apk are
1026 correct by looking at the build output - assume the metadata is
1027 correct. This takes away a useful level of sanity checking, and should
1028 only be used if the values can't be extracted.
1032 Another example, using extra parameters:
1034 @samp{Build Version:1.09.03,10903,45,subdir=Timeriffic,oldsdkloc=yes}
1037 @section AntiFeatures
1039 @cindex AntiFeatures
1041 This is optional - if present, it contains a comma-separated list of any of
1042 the following values, describing an anti-feature the application has.
1043 Even though such apps won't be displayed unless a settings box is ticked,
1044 it is a good idea to mention the reasons for the anti-feature(s) in the
1050 @samp{Ads} - the application contains advertising.
1053 @samp{Tracking} - the application tracks and reports your activity to
1054 somewhere without your consent. It's commonly used for when developers
1055 obtain crash logs without the user's consent, or when an app is useless
1056 without some kind of authentication.
1059 @samp{NonFreeNet} - the application relies on computational services that
1060 are impossible to replace or that the replacement cannot be connected to
1061 without major changes to the app.
1064 @samp{NonFreeAdd} - the application promotes non-Free add-ons, such that the
1065 app is effectively an advert for other non-free software and such software is
1066 not clearly labelled as such.
1069 @samp{NonFreeDep} - the application depends on a non-Free application (e.g.
1070 Google Maps) - i.e. it requires it to be installed on the device, but does not
1080 If this field is present, the application does not get put into the public
1081 index. This allows metadata to be retained while an application is temporarily
1082 disabled from being published. The value should be a description of why the
1083 application is disabled. No apks or source code archives are deleted: to purge
1084 an apk see the Build Version section or delete manually for developer builds.
1085 The field is therefore used when an app has outlived it's usefulness, because
1086 the source tarball is retained.
1089 @section Requires Root
1091 @cindex Requires Root
1093 Set this optional field to "Yes" if the application requires root
1094 privileges to be usable. This lets the client filter it out if the
1095 user so desires. Whether root is required or not, it is good to give
1096 a paragraph in the description to the conditions on which root may be
1097 asked for and the reason for it.
1099 @node Update Check Mode
1100 @section Update Check Mode
1102 @cindex Update Check Mode
1104 This determines the method using for determining when new releases are
1105 available - in other words, the updating of the Current Version and Current
1106 Version Code fields in the metadata by the @code{fdroid checkupdates} process.
1112 @code{None} - No checking is done because there's no appropriate automated way
1113 of doing so. Updates should be checked for manually. Use this, for example,
1114 when deploying betas or patched versions; when builds are done in a directory
1115 different to where the AndroidManifest.xml is; if the developers use the
1116 gradle build system and store version info in a separate file; if the
1117 developers make a new branch for each release and don't make tags; or if you've
1118 changed the package name or version code logic.
1120 @code{Static} - No checking is done - either development has ceased or new versions
1121 are not desired. This method is also used when there is no other checking method
1122 available and the upstream developer keeps us posted on new versions.
1124 @code{RepoManifest} - At the most recent commit, the AndroidManifest.xml file
1125 is looked for in the directory where it was found in the the most recent build.
1126 The appropriateness of this method depends on the development process used by
1127 the application's developers. You should not specify this method unless you're
1128 sure it's appropriate. For example, some developers bump the version when
1129 commencing development instead of when publishing.
1130 It will return an error if the AndroidManifest.xml has moved to a different
1131 directory or if the package name has changed.
1132 The current version that it gives may not be accurate, since not all
1133 versions are fit to be published. Therefore, before building, it is often
1134 necessary to check if the current version has been published somewhere by the
1135 upstream developers, either by checking for apks that they distribute or for
1136 tags in the source code repository.
1138 It currently works for every repository type to different extents, except
1139 the srclib repo type. For git, git-svn and hg repo types, you may use
1140 "RepoManifest/yourbranch" as UCM so that "yourbranch" would be the branch used
1141 in place of the default one. The default values are "master" for git,
1142 "default" for hg and none for git-svn (it stays in the same branch).
1143 On the other hand, branch support hasn't been implemented yet in bzr and svn,
1144 but RepoManifest may still be used without it.
1146 @code{RepoTrunk} - For svn and git-svn repositories, especially those who
1147 don't have a bundled AndroidManifest.xml file, the Tags and RepoManifest
1148 checks will not work, since there is no version information to obtain. But,
1149 for those apps who automate their build process with the commit ref that HEAD
1150 points to, RepoTrunk will set the Current Version and Current Version Code to
1153 @code{Tags} - The AndroidManifest.xml file in all tagged revisions in the
1154 source repository is checked, looking for the highest version code. The
1155 appropriateness of this method depends on the development process used by the
1156 application's developers. You should not specify this method unless you're sure
1157 it's appropriate. It shouldn't be used if the developers like to tag betas or
1158 are known to forget to tag releases. Like RepoManifest, it will not return the
1159 correct value if the directory containing the AndroidManifest.xml has moved.
1160 Despite these caveats, it is the often the favourite update check mode.
1162 It currently only works for git, hg, bzr and git-svn repositories. In the case
1163 of the latter, the repo URL must encode the path to the trunk and tags or else
1164 no tags will be found.
1166 @code{HTTP} - HTTP requests are used to determine the current version code and
1167 version name. This is controlled by the @code{Update Check Data} field, which
1168 is of the form @code{urlcode|excode|urlver|exver}.
1170 Firstly, if @code{urlcode} is non-empty, the document from that URL is
1171 retrieved, and matched against the regular expression @code{excode}, with the
1172 first group becoming the version code.
1174 Secondly, if @code{urlver} is non-empty, the document from that URL is
1175 retrieved, and matched against the regular expression @code{exver}, with the
1176 first group becoming the version name. The @code{urlver} field can be set to
1177 simply '.' which says to use the same document returned for the version code
1178 again, rather than retrieving a different one.
1181 @node Update Check Data
1182 @section Update Check Data
1184 @cindex Update Check Data
1186 Used in conjunction with @code{Update Check Mode} for certain modes.
1188 @node Vercode Operation
1189 @section Vercode Operation
1191 @cindex Vercode Operation
1193 Operation to be applied to the vercode obtained by the defined @code{Update
1194 Check Mode}. @code{%c} will be replaced by the actual vercode, and the whole
1195 string will be passed to python's @code{eval} function.
1197 Especially useful with apps that we want to compile for different ABIs, but
1198 whose vercodes don't always have trailing zeros. With @code{Vercode Operation}
1199 set at something like @code{%c*10 + 4}, we will be able to track updates and
1200 build three different versions of every upstream version.
1202 @node Archive Policy
1203 @section Archive Policy
1205 @cindex Archive Policy
1207 This determines the policy for moving old versions of an app to the archive
1208 repo, if one is configured. The configuration sets a default maximum number
1209 of versions kept in the main repo, after which older ones are moved to the
1210 archive. This app-specific policy setting can override that.
1212 Currently the only supported format is "n versions", where n is the number
1213 of versions to keep.
1215 @node Auto Update Mode
1216 @section Auto Update Mode
1218 @cindex Auto Update Mode
1220 This determines the method using for auto-generating new builds when new
1221 releases are available - in other words, adding a new Build Version line to the
1223 This happens in conjunction with the 'Update Check Mode' functionality - i.e.
1224 when an update is detected by that, it is also processed by this.
1230 @code{None} - No auto-updating is done
1232 @code{Version} - Identifies the target commit (i.e. tag) for the new build based
1233 on the given version specification, which is simply text in which %v and %c are
1234 replaced with the required version name and version code respectively.
1236 For example, if an app always has a tag "2.7.2" corresponding to version 2.7.2,
1237 you would simply specify "Version %v". If an app always has a tag "ver_1234"
1238 for a version with version code 1234, you would specify "Version ver_%c".
1240 Additionally, a suffix can be added to the version name at this stage, to
1241 differentiate F-Droid's build from the original. Continuing the first example
1242 above, you would specify that as "Version +-fdroid %v" - "-fdroid" is the suffix.
1246 @node Current Version
1247 @section Current Version
1249 @cindex Current Version
1251 The name of the version that is current. There may be newer versions of the
1252 application than this (e.g. betas), and there will almost certainly be older
1253 ones. This should be the one that is recommended for general use.
1254 In the event that there is no source code for the current version, or that
1255 non-free libraries are being used, this would ideally be the latest
1256 version that is still free, though it may still be expedient to
1257 retain the automatic update check — see No Source Since.
1259 This field is normally automatically updated - see Update Check Mode.
1261 This is converted to (@code{<marketversion>}) in the public index file.
1263 @node Current Version Code
1264 @section Current Version Code
1266 @cindex Current Version Code
1268 The version code corresponding to the Current Version field. Both these fields
1269 must be correct and matching although it's the current version code that's
1270 used by Android to determine version order and by F-Droid client to determine
1271 which version should be recommended.
1273 This field is normally automatically updated - see Update Check Mode.
1275 This is converted to (@code{<marketvercode>}) in the public index file.
1277 @node No Source Since
1278 @section No Source Since
1280 @cindex No Source Since
1282 In case we are missing the source code for the Current Version reported by
1283 Upstream, or that non-free elements have been introduced, this defines the
1284 first version that began to miss source code.
1285 Apps that are missing source code for just one or a few versions, but provide
1286 source code for newer ones are not to be considered here - this field is
1287 intended to illustrate which apps do not currently distribute source code, and
1288 since when have they been doing so.
1290 @node Update Processing
1291 @chapter Update Processing
1295 There are various mechanisms in place for automatically detecting that updates
1296 are available for applications, with the @code{Update Check Mode} field in the
1297 metadata determining which method is used for a particular application.
1299 Running the @code{fdroid checkupdates} command will apply this method to each
1300 application in the repository and update the @code{Current Version} and
1301 @code{Current Version Code} fields in the metadata accordingly.
1303 As usual, the @code{-p} option can be used with this, to restrict processing
1304 to a particular application.
1306 Note that this only updates the metadata such that we know what the current
1307 published/recommended version is. It doesn't make that version available in
1308 the repository - for that, see the next section.
1312 Adding updates (i.e. new versions of applications already included in the
1313 repository) happens in two ways. The simple case is applications where the
1314 APK files are binaries, retrieved from a developer's published build. In this
1315 case, all that's required is to place the new binary in the @code{Repo}
1316 directory, and the next run of @code{fdroid update} will pick it up.
1318 For applications built from source, it is necessary to add a new
1319 @code{Build Version} line to the metadata file. At the very least, the version
1320 name, version code and commit will be different. It is also possible that the
1321 additional build flags will change between versions.
1323 For processing multiple updates in the metadata at once, it can be useful to
1324 run @code{fdroid update --interactive}. This will check all the applications
1325 in the repository, and where updates are required you will be prompted to
1326 [E]dit the metadata, [I]gnore the update, or [Q]uit altogether.
1329 @chapter Build Server
1331 The Build Server system isolates the builds for each package within a clean,
1332 isolated and secure throwaway virtual machine environment.
1336 Building applications in this manner on a large scale, especially with the
1337 involvement of automated and/or unattended processes, could be considered
1338 a dangerous pastime from a security perspective. This is even more the case
1339 when the products of the build are also distributed widely and in a
1340 semi-automated ("you have updates available") fashion.
1342 Assume that an upstream source repository is compromised. A small selection
1343 of things that an attacker could do in such a situation:
1347 Use custom ant build steps to execute virtually anything as the user doing
1350 Access the keystore.
1352 Modify the built apk files or source tarballs for other applications in the
1355 Modify the metadata (which includes build scripts, which again, also includes
1356 the ability to execute anything) for other applications in the repository.
1359 Through complete isolation, the repurcussions are at least limited to the
1360 application in question. Not only is the build environment fresh for each
1361 build, and thrown away afterwards, but it is also isolated from the signing
1364 Aside from security issues, there are some applications which have strange
1365 requirements such as custom versions of the NDK. It would be impractical (or
1366 at least extremely messy) to start modifying and restoring the SDK on a
1367 multi-purpose system, but within the confines of a throwaway single-use
1368 virtual machine, anything is possible.
1370 All this is in addition to the obvious advantage of having a standardised
1371 and completely reproducible environment in which builds are made. Additionally,
1372 it allows for specialised custom build environments for particular
1375 @section Setting up a build server
1377 In addition to the basic setup previously described, you will also need
1378 a Vagrant-compatible Debian Testing base box called 'testing32' (or testing64
1379 for a 64-bit VM, if you want it to be much slower, and require more disk
1382 You can use a different version or distro for the base box, so long as you
1383 don't expect any help making it work. One thing to be aware of is that
1384 working copies of source trees are moved from the host to the guest, so
1385 for example, having subversion v1.6 on the host and v1.7 on the guest
1388 Unless you're very trusting. you should create one of these for yourself
1389 from verified standard Debian installation media. However, you could skip
1390 over the next few paragraphs (and sacrifice some security) by downloading
1391 @url{https://f-droid.org/testing32.box}.
1393 Documentation for creating a base box can be found at
1394 @url{http://docs.vagrantup.com/v1/docs/base_boxes.html}.
1396 In addition to carefully following the steps described there, you should
1397 consider the following:
1401 It is advisable to disable udev network device persistence, otherwise any
1402 movement of the VM between machines, or reconfiguration, will result in
1405 For a Debian/Ubuntu default install, just
1406 @code{touch /etc/udev/rules.d/75-persistent-net-generator.rules} to turn
1407 off rule generation, and at the same time, get rid of any rules it's
1408 already created in @code{/etc/udev/rules.d/70-persistent-net.rules}.
1410 Unless you want the VM to become totally inaccessible following a failed
1411 boot, you need to set @code{GRUB_RECORDFAIL_TIMEOUT} to a value other than
1412 -1 in @code{/etc/grub/default} and then run @code{update-grub}.
1416 With this base box available, you should then create @code{makebs.config.py},
1417 using @code{makebs.config.sample.py} as a reference - look at the settings and
1418 documentation there to decide if any need changing to suit your environment.
1419 There is a path for retrieving the base box if it doesn't exist, and an apt
1420 proxy definition, both of which may need customising for your environment.
1421 You can then go to the @code{fdroidserver} directory and run this:
1427 This will take a long time, and use a lot of bandwidth - most of it spent
1428 installing the necessary parts of the Android SDK for all the various
1429 platforms. Luckily you only need to do it occasionally. Once you have a
1430 working build server image, if the recipes change (e.g. when packages need
1431 to be added) you can just run that script again and the existing one will
1432 be updated in place.
1434 The main sdk/ndk downloads will automatically be cached to speed things
1435 up the next time, but there's no easy way of doing this for the longer
1436 sections which use the SDK's @code{android} tool to install platforms,
1437 add-ons and tools. However, instead of allowing automatic caching, you
1438 can supply a pre-populated cache directory which includes not only these
1439 downloads, but also .tar.gz files for all the relevant additions. If the
1440 provisioning scripts detect these, they will be used in preference to
1441 running the android tools. For example, if you have
1442 @code{buildserver/addons/cache/platforms/android-19.tar.gz} that will be
1443 used when installing the android-19 platform, instead of re-downloading it
1444 using @code{android update sdk --no-ui -t android-19}.
1446 Once it's complete you'll have a new base box called 'buildserver' which is
1447 what's used for the actual builds. You can then build packages as normal,
1448 but with the addition of the @code{--server} flag to @code{fdroid build} to
1449 instruct it to do all the hard work within the virtual machine.
1451 The first time a build is done, a new virtual machine is created using the
1452 'buildserver' box as a base. A snapshot of this clean machine state is saved
1453 for use in future builds, to improve performance. You can force discarding
1454 of this snapshot and rebuilding from scratch using the @code{--resetserver}
1455 switch with @code{fdroid build}.
1460 There are two kinds of signing involved in running a repository - the signing
1461 of the APK files generated from source builds, and the signing of the repo
1462 index itself. The latter is optional, but very strongly recommended.
1464 @section Repo Index Signing
1466 When setting up the repository, one of the first steps should be to generate
1467 a signing key for the repository index. This will also create a keystore, which
1468 is a file that can be used to hold this and all other keys used. Consider the
1469 location, security and backup status of this file carefully, then create it as
1472 @code{keytool -genkey -v -keystore my.keystore -alias repokey -keyalg RSA -keysize 2048 -validity 10000}
1474 In the above, replace 'my.keystore' with the name of the keystore file to be
1475 created, and 'repokey' with a name to identify the repo index key by.
1477 You'll be asked for a password for the keystore, AND a password for the key.
1478 They shouldn't be the same. In between, you'll be asked for some identifying
1479 details which will go in the certificate.
1481 The two passwords entered go into @code{config.py}, as @code{keystorepass} and
1482 @code{keypass} respectively. The path to the keystore file, and the alias you
1483 chose for the key also go into that file, as @code{keystore} and
1484 @code{repo_keyalias} respectively.
1486 @section Package Signing
1488 With the repo index signing configured, all that remains to be done for package
1489 signing to work is to set the @code{keydname} field in @code{config.py} to
1490 contain the same identifying details you entered before.
1492 A new key will be generated using these details, for each application that is
1493 built. (If a specific key is required for a particular application, this system
1494 can be overridden using the @code{keyaliases} config settings.
1497 @node GNU Free Documentation License
1498 @appendix GNU Free Documentation License