Differences in multithreading between Python and Jython

It seems that two or more threads do not work in parallel at the same time due to the influence of GIL (Global Interpreter Lock) in threading.Thread of Python (CPython) (although it works in parallel). Then, I wondered what about other implementations, so I checked it with Jython. Also, Jython can use Java API, so I checked java.lang.Thread as well.

Verification environment

Source code for verification

Create a worker that distributes and calculates tasks that enumerate prime numbers in the range of 4 to 100,000. The following is when threading.Thread is used.

py_worker.py


from threading import Thread

class Worker(Thread):
    def __init__(self, start, end):
        super(Worker, self).__init__()
        self._start = start
        self._end = end

    def run(self):
        self.prime_nums = []
        for i in xrange(self._start, self._end):
            if not 0 in self._remainders(i):
                self.prime_nums.append(i)

    def _remainders(self, end, start=2):
        for i in xrange(start, end):
            yield end % i

The following is when using java.lang.Thread. (Only the class to import is different)

jy_worker.py


from java.lang import Thread

class Worker(Thread):
    def __init__(self, start, end):
        super(Worker, self).__init__()
        self._start = start
        self._end = end

    def run(self):
        self.prime_nums = []
        for i in xrange(self._start, self._end):
            if not 0 in self._remainders(i):
                self.prime_nums.append(i)

    def _remainders(self, end, start=2):
        for i in xrange(start, end):
            yield end % i

And the process of kicking the worker thread and measuring the elapsed time is as follows.

main.py


import sys
from threading import Thread
from datetime import datetime

def total_seconds(td):
    return (td.microseconds + (td.seconds + td.days * 24 * 3600) * 10**6) / 10**6

if __name__ == '__main__':
    argv = sys.argv
    argc = len(argv)
    if argc < 4:
        print 'ERROR: <worker_module> <n_workers> <max_value>'
        sys.exit(1)
    worker_module = argv[1]
    n_workers = int(argv[2])
    max_value = int(argv[3])
    min_value = 4
    interval = (max_value - min_value) / n_workers
    Worker = __import__(worker_module).Worker

    workers = []
    for start in xrange(4, max_value, interval):
        print 'Worker: %s, %s' % (start, start+interval)
        worker = Worker(start, start+interval)
        workers.append(worker)

    start_time = datetime.utcnow()
    for worker in workers:
        worker.start()
    for worker in workers:
        worker.join()
    end_time = datetime.utcnow()
    elapsed_time = end_time - start_time
    elapsed_sec = total_seconds(elapsed_time)
    n_primes = sum([len(w.prime_nums) for w in workers])
    print '# of primes = %s, time = %s sec' % (n_primes, elapsed_sec)

result

The elapsed time until the worker processing is completed is as follows.

Implementation class 1 thread 2 threads
Python threading.Thread 100 sec 125 sec
Jython threading.Thread 101 sec 73 sec
Jython java.lang.Thread 101 sec 77 sec

Python can only run one thread at a time, so distributing it across two threads doesn't make it faster (rather slower), but Jython gave different results.

Since the elapsed time in one thread is almost the same in Python and Jython, I think that the basic performance for the processing used this time will not change. (Actually, I expected that Java dynamic compilation would work and Jython would be faster). And in the case of Jython, 2 threads finished earlier than 1 thread, so it feels like they are working in parallel. I wonder if the behavior here is implementation-dependent.

Recommended Posts

Differences in multithreading between Python and Jython
Differences in authenticity between Python and JavaScript
Differences between Ruby and Python in scope
Differences in syntax between Python and Java
Difference between list () and [] in Python
Difference between == and is in python
Differences between Python, stftime and strptime
difference between statements (statements) and expressions (expressions) in Python
Difference between @classmethod and @staticmethod in Python
Difference between append and + = in Python list
Difference between nonlocal and global in Python
Differences between Ruby and Python (basic syntax)
Differences between queryStringParameters and multiValueQueryStringParameters in AWS Lambda
Summary of the differences between PHP and Python
[python] Difference between variables and self. Variables in class
About the difference between "==" and "is" in python
Differences in string processing between Python, Ruby, JS, PHP (combination and variable expansion)
Difference between Ruby and Python in terms of variables
Calculate Pose and Transform differences in Python with ROS
Stack and Queue in Python
Mutual conversion between JSON and YAML / TOML in Python
Difference between return, return None, and no return description in Python
Unittest and CI in Python
Compare "relationship between log and infinity" in Gauche (0.9.4) and Python (3.5.1)
Differences in behavior between append () and "+ =" operators when adding data to a list in Python
Python module num2words Difference in behavior between English and Russian
List concatenation method in python, difference between list.extend () and “+” operator
I tried to enumerate the differences between java and python
Difference between Ruby and Python split
Differences between Windows and Linux directories
Difference between java and python (memo)
MIDI packages in Python midi and pretty_midi
View photos in Python and html
Sorting algorithm and implementation in Python
Differences between symbolic links and hard links
Manipulate files and folders in Python
About dtypes in Python and Cython
Assignments and changes in Python objects
Cooperation between python module and API
Difference between python2 series and python3 series dict.keys ()
Ciphertext in Python: IND-CCA2 and RSA-OAEP
Hashing data in R and Python
Function synthesis and application in Python
Export and output files in Python
[Python] Difference between function and method
Reverse Hiragana and Katakana in Python2.7
Reading and writing text in Python
[GUI in Python] PyQt5-Menu and Toolbar-
Python --Difference between exec and eval
[Python] Difference between randrange () and randint ()
Differences in identity, equivalence, and aliases
Differences between Python, read (), readline (), readlines ()
[Python] Difference between sorted and sorted (Colaboratory)
Create and read messagepacks in Python
Difference in how to write if statement between ruby ​​and python
Transcendental simple and clear! !! Difference between single quotes and double quotes in Python
Summary of differences between Python and PHP (comparison table of main items)
File open function in Python3 (difference between open and codecs.open and speed comparison)
Communicate between Elixir and Python with gRPC
Overlapping regular expressions in Python and Java
Notes using cChardet and python3-chardet in Python 3.3.1.