- cross-posted to:
- [email protected]
- cross-posted to:
- [email protected]
You must log in or register to comment.
I’m curious to see how this whole thing shakes out. Like, will removing the GIL be an uphill battle that everyone regrets even suggesting?Will it be so easy, we wonder why we didn’t do it years ago? Or, most likely, somewhere in the middle?
Did you read the article?
Yes, testing infrastructure is being put in place and some low-hanging fruit bugs have already been squashed. This bodes well, but it’s still early days, and I imagine not a lot of GIL-less production deployments are out there yet - where the real showstoppers will potentially live.
I’m tenatively optimistic, but threading bugs are sometimes hard to catch
threading bugs are sometimes hard to catch
Putting it mildly! Threading bugs are probably the worst class of bugs to debug
Definitely debatable if this is worth the risk of impossible bugs. Python is very slow, and multi threading isn’t going to change that. 4x extremely slow is still extremely slow. If you care remotely about performance you need to use a different language anyway.
Python can be extremely slow, it doesn’t have to be. I recently re-wrote a stats program at work and got a ~500x speedup over the original python and a 10x speed up over the c++ rewrite of that. If you know how python works and avoid the performance foot-guns like nested loops you can often (though not always) get good performance.
Unless the C++ code was doing something wrong there’s literally no way you can write pure Python that’s 10x faster than it. Something else is going on there. Maybe the c++ code was accidentally O(N^2) or something.
In general Python will be 10-200 times slower than C++. 50x slower is typical.
Nope, if you’re working on large arrays of data you can get significant speed ups using well optimised BLAS functions that are vectorised (numpy) which beats out simply written c++ operating on each array element in turn. There’s also Numba which uses LLVM to jit compile a subset of python to get compiled performance, though I didnt go to that in this case.
You could link the BLAS libraries to c++ but its significantly more work than just importing numpy from python.
numpy
Numpy is written in C.
Numba
Numba is interesting… But a) it can already do multithreading so this change makes little difference, and b) it’s still not going to be as fast as C++ (obviously we don’t count the GPU backend).
It’s exciting, but man there are lots of assumptions in native python built around the gil.
I’ve seen lists, etc. modified by threads assuming the gil locks for them. Testing this e2e for any production deployment can be a bit of a nightmare.
My company makes it super easy for me - we’re just going to continue on python 2.7 and add this to the long list of reasons why we’re not upgrading.
Please send help.
Python 2.7 and iOS mobile programmers stuck on Objective-C could start a support group.
You may be pleased to know that PyPy’s Python 2.7 branch will be maintained indefinitely, since PyPy is also written in Python 2.7. Also, if you can’t leave CPython yet, ActivePython’s team is publishing CPython 2.7 security patches.
We already have contracts in place to get security patches. That’s usually the InfoSec team’s problem anyway.
As a developer, my life gets hard due to library support. We manage internal forks of multiple open source projects just to make them python 2 compatible. A non-trivial amount of time is wasted on this, and we don’t even have it available for public use. 🤷♂️
