| Store | Cart

Re: [Python-Dev] [Speed] Benchmarks: Comparison between Python 2.7 and Python 3.6 performance

From: Miquel Torres <tob...@gmail.com>
Fri, 04 Nov 2016 19:18:48 +0000
Nice! For the record, I'll be giving a talk in PyCon Ireland about
Codespeed. Would you mind me citing those tweets and screenshots, to
highlight usage on speed.python.org?

You mentioned new more reliable vs old results. How close are we to have an
stable setup that gives us benchmarks numbers regularly?

Cheers,
Miquel
El El vie, 4 nov 2016 a las 12:30, Victor Stinner <vict...@gmail.com>
escribió:

> Hi,>> You may know that I'm working on benchmarks. I regenerated all> benchmark results of speed.python.org using performance 0.3.2> (benchmark suite). I started to analyze results.>> All results are available online on the website:>>    https://speed.python.org/>>> To communicate on my work on benchmarks, I tweeted two pictures:>> "sympy benchmarks: Python 3.6 is between 8% and 48% faster than Python> 2.7 #python #benchmark":> https://twitter.com/VictorStinner/status/794289596683210760>> "Python 3.6 is between 25% and 54% slower than Python 2.7 in the> following benchmarks":> https://twitter.com/VictorStinner/status/794305065708376069>>> Many people were disappointed that Python 3.6 can be up to 54% slower> than Python 2.7. In fact, I know many reasons which explain that, but> it's hard to summarize them in 140 characters ;-)>> For example, Python 3.6 is 54% slower than Python 2.7 on the benchmark> pycrypto_aes. This benchmark tests a pure Python implementation of the> crypto cipher AES. You may know that CPython is slow for CPU intensive> functions, especially on integer and floatting point numbers.>> "int" in Python 3 is now "long integers" by default, which is known to> be a little bit slower than "short int" of Python 2. On a more> realistic benchmark (see other benchmarks), the overhead of Python 3> "long int" is negligible.>> AES is a typical example stressing integers. For me, it's a dummy> benchmark: it doesn't make sense to use Python for AES: modern CPUs> have an *hardware* implemention which is super fast.>>> Well, I didn't have time to analyze in depth individual benchmarks. If> you want to help me, here is the source code of benchmarks:> https://github.com/python/performance/blob/master/performance/benchmarks/>>> Raw results of Python 3.6 compared to Python 2.7:> -------------------> $ python3 -m perf compare_to 2016-11-03_15-36-2.7-91f024fc9b3a.json.gz> 2016-11-03_15-38-3.6-c4319c0d0131.json.gz -G --min-speed=5> Slower (40):> - python_startup: 7.74 ms +- 0.28 ms -> 26.9 ms +- 0.6 ms: 3.47x slower> - python_startup_no_site: 4.43 ms +- 0.08 ms -> 10.4 ms +- 0.4 ms: 2.36x> slower> - unpickle_pure_python: 417 us +- 3 us -> 918 us +- 14 us: 2.20x slower> - call_method: 16.3 ms +- 0.2 ms -> 28.6 ms +- 0.8 ms: 1.76x slower> - call_method_slots: 16.2 ms +- 0.4 ms -> 28.3 ms +- 0.7 ms: 1.75x slower> - call_method_unknown: 18.4 ms +- 0.2 ms -> 30.8 ms +- 0.8 ms: 1.67x slower> - crypto_pyaes: 161 ms +- 2 ms -> 249 ms +- 2 ms: 1.54x slower> - xml_etree_parse: 201 ms +- 5 ms -> 298 ms +- 8 ms: 1.49x slower> - logging_simple: 26.4 us +- 0.3 us -> 38.4 us +- 0.7 us: 1.46x slower> - logging_format: 31.3 us +- 0.4 us -> 45.5 us +- 0.8 us: 1.45x slower> - pickle_pure_python: 986 us +- 9 us -> 1.41 ms +- 0.03 ms: 1.43x slower> - spectral_norm: 208 ms +- 2 ms -> 287 ms +- 2 ms: 1.38x slower> - logging_silent: 660 ns +- 7 ns -> 865 ns +- 31 ns: 1.31x slower> - chaos: 240 ms +- 2 ms -> 314 ms +- 4 ms: 1.31x slower> - go: 490 ms +- 2 ms -> 640 ms +- 26 ms: 1.31x slower> - xml_etree_iterparse: 178 ms +- 2 ms -> 230 ms +- 5 ms: 1.29x slower> - sqlite_synth: 8.29 us +- 0.16 us -> 10.6 us +- 0.2 us: 1.28x slower> - xml_etree_process: 210 ms +- 6 ms -> 268 ms +- 14 ms: 1.28x slower> - django_template: 387 ms +- 4 ms -> 484 ms +- 5 ms: 1.25x slower> - fannkuch: 830 ms +- 32 ms -> 1.04 sec +- 0.03 sec: 1.25x slower> - hexiom: 20.2 ms +- 0.1 ms -> 24.7 ms +- 0.2 ms: 1.22x slower> - chameleon: 26.1 ms +- 0.2 ms -> 31.9 ms +- 0.4 ms: 1.22x slower> - regex_compile: 395 ms +- 2 ms -> 482 ms +- 6 ms: 1.22x slower> - json_dumps: 25.8 ms +- 0.2 ms -> 31.0 ms +- 0.5 ms: 1.20x slower> - nqueens: 229 ms +- 2 ms -> 274 ms +- 2 ms: 1.20x slower> - genshi_text: 81.9 ms +- 0.6 ms -> 97.8 ms +- 1.1 ms: 1.19x slower> - raytrace: 1.17 sec +- 0.03 sec -> 1.39 sec +- 0.03 sec: 1.19x slower> - scimark_monte_carlo: 240 ms +- 7 ms -> 282 ms +- 10 ms: 1.17x slower> - scimark_sor: 441 ms +- 8 ms -> 517 ms +- 12 ms: 1.17x slower> - deltablue: 17.4 ms +- 0.1 ms -> 20.1 ms +- 0.6 ms: 1.16x slower> - sqlalchemy_declarative: 310 ms +- 3 ms -> 354 ms +- 6 ms: 1.14x slower> - call_simple: 12.2 ms +- 0.2 ms -> 13.9 ms +- 0.2 ms: 1.14x slower> - scimark_fft: 613 ms +- 19 ms -> 694 ms +- 23 ms: 1.13x slower> - meteor_contest: 191 ms +- 1 ms -> 215 ms +- 2 ms: 1.13x slower> - pathlib: 46.9 ms +- 0.4 ms -> 52.6 ms +- 0.9 ms: 1.12x slower> - richards: 181 ms +- 1 ms -> 201 ms +- 6 ms: 1.11x slower> - genshi_xml: 191 ms +- 2 ms -> 209 ms +- 2 ms: 1.10x slower> - float: 290 ms +- 5 ms -> 310 ms +- 7 ms: 1.07x slower> - scimark_sparse_mat_mult: 8.19 ms +- 0.22 ms -> 8.74 ms +- 0.15 ms:> 1.07x slower> - xml_etree_generate: 302 ms +- 3 ms -> 320 ms +- 8 ms: 1.06x slower>> Faster (15):> - telco: 707 ms +- 22 ms -> 22.1 ms +- 0.4 ms: 32.04x faster> - unpickle_list: 15.0 us +- 0.3 us -> 7.86 us +- 0.16 us: 1.90x faster> - pickle_list: 14.7 us +- 0.2 us -> 9.12 us +- 0.38 us: 1.61x faster> - json_loads: 98.7 us +- 2.3 us -> 62.3 us +- 0.7 us: 1.58x faster> - pickle: 40.4 us +- 0.6 us -> 27.1 us +- 0.5 us: 1.49x faster> - sympy_sum: 361 ms +- 10 ms -> 244 ms +- 7 ms: 1.48x faster> - sympy_expand: 1.68 sec +- 0.02 sec -> 1.15 sec +- 0.03 sec: 1.47x faster> - regex_v8: 62.0 ms +- 0.5 ms -> 47.2 ms +- 0.6 ms: 1.31x faster> - sympy_str: 699 ms +- 22 ms -> 537 ms +- 15 ms: 1.30x faster> - regex_effbot: 6.67 ms +- 0.04 ms -> 5.23 ms +- 0.05 ms: 1.28x faster> - mako: 61.5 ms +- 0.7 ms -> 49.7 ms +- 2.5 ms: 1.24x faster> - html5lib: 298 ms +- 7 ms -> 261 ms +- 6 ms: 1.14x faster> - sympy_integrate: 55.9 ms +- 0.3 ms -> 51.8 ms +- 1.0 ms: 1.08x faster> - pickle_dict: 69.4 us +- 0.9 us -> 65.2 us +- 3.2 us: 1.06x faster> - scimark_lu: 551 ms +- 26 ms -> 523 ms +- 18 ms: 1.05x faster>> Benchmark hidden because not significant (8): 2to3, dulwich_log,> nbody, pidigits, regex_dna, tornado_http, unpack_sequence, unpickle> Ignored benchmarks (3) of 2016-11-03_15-36-2.7-91f024fc9b3a.json:> hg_startup, pyflate, spambayes> ------------------->> Please ignore call_method, call_method_slots, call_method_unknown> benchmarks: it seems like I had an issue on the benchmark server. I> was unable to reproduce he 70% slowdown on my laptop.>> I attached the two compressed JSON files to this email if you want to> analyze them yourself.>> I hope that my work on benchmarks will motive some developers to look> closer at Python 3 performance to find interesting optimizations ;-)>> Victor> _______________________________________________> Speed mailing list> Spe...@python.org> https://mail.python.org/mailman/listinfo/speed>

_______________________________________________
Python-Dev mailing list
Pyth...@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/python-dev-ml%40activestate.com

Recent Messages in this Thread
Victor Stinner Nov 04, 2016 12:28 pm
Miquel Torres Nov 04, 2016 07:18 pm
Victor Stinner Nov 04, 2016 08:56 pm
Messages in this thread