Merge branch 'asynchronous_reader' into 'master'

Add asynchronous filereader, fix python3 lockups

with the current implementation of AsynchronousFileReader with Python 3 FDroidPopen deadlocks with 100% CPU-utilization

The code is from https://github.com/soxofaan/asynchronousfilereader

I hope the MIT-Licence makes no problems

See merge request !80

diff --git a/docs/fdroid.texi b/docs/fdroid.texi
index af58c0bb2ef73da6933ae25417c097005e8a599e..402a8133024899c926d88dd415667b6dc961e97d 100644 (file)
--- a/docs/fdroid.texi
+++ b/docs/fdroid.texi
@@ -927,7 +927,7 @@ the revision or tag to use in the respective source control.
 For Ant projects, you can optionally append a number with a colon at the
 beginning of a srclib item to automatically place it in project.properties as
 a library under the specified number. For example, if you specify
-@code{1:somelib@@1.0}, f-droid will automatically do the equivalent of the
+@code{1:somelib@@1.0}, F-Droid will automatically do the equivalent of the
 legacy practice @code{prebuild=echo "android.library.reference.1=$$somelib$$"
 >> project.properties}.
 
@@ -941,6 +941,31 @@ update the project with a particular target. You can then also use $$name$$ in
 the init/prebuild/build command to substitute the relative path to the library
 directory, but it could need tweaking if you've changed into another directory.
 
+Currently srclibs are necessary when upstream uses jar files or pulls
+dependencies from non-trusted repositories. While there is no guarantee that
+those binaries are free and correspondent to the source code, F-Droid allows 
+the following known repositories until a source-built alternative is available:
+
+@itemize @bullet
+
+@item
+@samp{mavenCentral} - the original repo, hardcoded in Maven and Gradle.
+
+@item
+@samp{jCenter} - hardcoded in Gradle, this repo by Bintray tries to provide
+easier handling. It should sync with mavenCentral from time to time.
+
+@item
+@samp{OSS Sonatype} - maintained by the people behind mavenCentral, this
+repository focuses on hosting services for open source project binaries.
+
+@item
+@samp{JitPack.io} - builds directly from Github repositories. However,
+they do not provide any option to reproduce or verify the resulting
+binaries. Builds pre-release versions in some cases.
+
+@end itemize
+
 @item patch=x
 Apply patch(es). 'x' names one (or more - comma-seperated) files within a
 directory below the metadata, with the same name as the metadata file but
@@ -1034,7 +1059,7 @@ used. Note that for projects with flavours, you must specify at least one
 valid flavour since 'yes' or 'main' will build all of them separately.
 
 @item maven=yes[@@<dir>]
-Build with Maven instead of Ant. An extra @@<dir> tells f-droid to run Maven
+Build with Maven instead of Ant. An extra @@<dir> tells F-Droid to run Maven
 inside that relative subdirectory. Sometimes it is needed to use @@.. so that
 builds happen correctly.
 

diff --git a/fdroidserver/scanner.py b/fdroidserver/scanner.py
index f46627d38cab0539bc88e9f5e5ac951b5ac0ee0a..9ebfdb17478ac03bdbe9417c8a3c37cf476eff9d 100644 (file)
--- a/fdroidserver/scanner.py
+++ b/fdroidserver/scanner.py
@@ -31,6 +31,15 @@ config = None
 options = None
 
 
+def get_gradle_compile_commands(thisbuild):
+    compileCommands = ['compile', 'releaseCompile']
+    if thisbuild['gradle'] != ['yes']:
+        compileCommands += [flavor + 'Compile' for flavor in thisbuild['gradle']]
+        compileCommands += [flavor + 'ReleaseCompile' for flavor in thisbuild['gradle']]
+
+    return [re.compile(r'\s*' + c, re.IGNORECASE) for c in compileCommands]
+
+
 # Scan the source code in the given directory (and all subdirectories)
 # and return the number of fatal problems encountered
 def scan_source(build_dir, root_dir, thisbuild):
@@ -114,6 +123,11 @@ def scan_source(build_dir, root_dir, thisbuild):
             d = f.read(1024)
         return bool(d.translate(None, textchars))
 
+    gradle_compile_commands = get_gradle_compile_commands(thisbuild)
+
+    def is_used_by_gradle(line):
+        return any(command.match(line) for command in gradle_compile_commands)
+
     # Iterate through all files in the source code
     for r, d, f in os.walk(build_dir, topdown=True):
 
@@ -157,8 +171,9 @@ def scan_source(build_dir, root_dir, thisbuild):
                     continue
                 for i, line in enumerate(file(fp)):
                     i = i + 1
-                    for name in suspects_found(line):
-                        count += handleproblem('usual supect \'%s\' at line %d' % (name, i), fd, fp)
+                    if is_used_by_gradle(line):
+                        for name in suspects_found(line):
+                            count += handleproblem('usual supect \'%s\' at line %d' % (name, i), fd, fp)
 
             elif ext in ['', 'bin', 'out', 'exe']:
                 if is_binary(fp):