Google App Engine python 2.7 runtime includes support for numpy and matplotlib since 13-Dec-2012, however, by default, matplotlib is not supported on the development server. This workaround let you run both in developer mode and deployed on google app engine:
Download
Copy to clipboard1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | #!/usr/bin/python
import logging
try:
import webapp2
except:
logging.exception("no webapp")
import pprint
import os
import StringIO
os.environ["MATPLOTLIBDATA"] = os.getcwdu()
os.environ["MPLCONFIGDIR"] = os.getcwdu()
import subprocess
def no_popen(*args, **kwargs): raise OSError("forbjudet")
subprocess.Popen = no_popen
subprocess.PIPE = None
subprocess.STDOUT = None
logging.warn("E: %s" % pprint.pformat(os.environ))
try:
import numpy, matplotlib, matplotlib.pyplot as plt
except:
logging.exception("trouble")
def dynamic_png():
try:
plt.title("Dynamic PNG")
for i in range(5): plt.plot(sorted(numpy.random.randn(25)))
rv = StringIO.StringIO()
plt.savefig(rv, format="png")
plt.clf()
return """<img src="data:image/png;base64,%s"/>""" % rv.getvalue().encode("base64").strip()
finally:
plt.clf()
def dynamic_svg():
try:
plt.title("Dynamic SVG")
for i in range(5): plt.plot(sorted(numpy.random.randn(25)))
rv = StringIO.StringIO()
plt.savefig(rv, format="svg")
return rv.getvalue()
finally:
plt.clf()
if __name__ == "__main__":
print dynamic_png()
print dynamic_svg()
else:
class MainHandler(webapp2.RequestHandler):
def get(self):
self.response.write("""<html><head/><body>""")
self.response.write(dynamic_png())
self.response.write(dynamic_svg())
self.response.write("""</body> </html>""")
app = webapp2.WSGIApplication([
('/', MainHandler)
], debug=True)
|
Copy contents of mpl-data to the app directory. When you run main.py from command line, from app directory, matplotlib will cache font info, etc, in config dir, which is also set to app directory. This caching can only be done locally, it won't work when deployed, though, hopefully, the cache snapshot is used (?). Live demo at http://gae-matplotlib-demo.appspot.com/ TeX won't work though.
$ cat app.yaml application: gae-matplotlib-demo version: 1 runtime: python27 api_version: 1 threadsafe: no
handlers: - url: /favicon.ico static_files: favicon.ico upload: favicon.ico
- url: .* script: main.app
libraries:
- name: webapp2 version: "2.5.2"
- name: numpy version: "latest"
- name: matplotlib version: "latest"
skip_files: - .xxxx-stupud-defaults
also copy dateutil into app; somehow gae's version is missing rrule.
Hi Dima,
I'm the Google developer in charge of packaging Matplotlib on App Engine, and we're very glad you took interest in making it work better and sharing it publicly. Thanks for doing this.
Unfortunately, we could not easily get the local version of Matplotlib to not attempt to write files and other GAE-prohibited activities. We'd like to eventually make it possible to run Matplotlib on the dev appserver without any effort, but in the meantime, this seems like an acceptable workaround.
I ran into a few points of confusion with your instructions:
matplotlibrcin the app directory, and changebackendfromGTKAggtoAgg.python main.pyfirst, i.e., just run the script directly in Python, NOT in the dev appserver. After you have done that once, you can run it in the dev appserver.Then it worked! Fantastic.
Unfortunately, it won't. We've disabled both writing and reading from the cache on production. I don't think that it makes sense to use a font cache you generated on your machine in the production App Engine, because it will have references to other fonts on your local system.
Can you clarify this with an example? We haven't removed rrule, and I don't see any obvious problems (although I haven't extensively looked into the date handling).
Thanks again,
Matt
Hey Matt,
maybe, somehow it worked with and without in my install
that's the whole idea, run it once from command line and it gens all the required stuff, sortof.
I'm not enitrely sure if it does or doesn't reference system fonts, on one hand, matplotlibs own fonts are copied over and fontconfig vis os.popen is blocked, so presumable system fonts are not cached, on the other I have no clue what weird format the cache uses.
still somehow matplotlib appears to notice invalid cache and tries to recreate is, which is why try: plot_smth() / except: pass is necessary at app startup.
dateutil.rrule is somehow missing in gae, at least I got an import error, if it was not removed, then perhaps it's not in the magic list of allowed modules? it is available in system python 2.7.3 and it is used by matplotlib. I'm not entirely sure about this now, I think the import error was matplotlib trying to import dateutils.rrule and failing, I don't recall if that was against dev server or production.
I may have been too fast on some of the hacks, there surely were bits I missed or misunderstood, I just wanted to finish this quick for someone.
Hey,
Agg, which is what it will be by default on production.Good point -- it probably can't access system fonts since you replaced
subprocess.Popen. We could theoretically enable cache reading (but not writing) on production, but it would be a low priority for us.As for dateutil.rrule: ah, I see you are trying to directly import it yourself ("
from dateutil import rrule"). We do not currently provide the top-level dateutil package on App Engine. Matplotlib has access to its own internal version of dateutil, but this is not intended for use outside of Matplotlib.Is it necessary to manually import dateutil for ordinary use with Matplotlib? If so, we probably need to change how we make dateutil available. The examples I have seen (e.g., http://matplotlib.org/examples/pylab_examples/date_demo_rrule.html) do not directly use dateutil. If you need to use dateutil for non-Matplotlib-related purposes, then our current advice is to include it with your app.
Hey, get in touch on g+.
Hi Dima. Can you put a listing of all the files that are in your test app's directory? I am trying to make this work and failing miserably. Thanks, Andrew
Andrew, sorry I don't have a local copy of the source for exact listing anymore. I guess I could extract it from live demo somehow. Fundamentally I ran it locally under strace and checked what files were actually opened.
Hi Dima. I'm hoping for a little clarification on this. I've been trying to get things working locally on a Mac but can't import matplotlib when running GAE locally.
Did you add a copy of the matplotlib module within your GAE project directory (e.g. under a lib folder) and import it from there when running GAE localy, or were you able to import it from the OS module path where it is imported from when running Python outside of GAE?
@David I don't think I have sources from that little project any longer.
Please re-read recipe text carefully, at that time it worked fine, the idea was to dynamically cache matplotlib's files into the project by running this python file from command line without GAE, then package it all into a GAE app.