Popular Python recipes tagged "meta:requires=urllib2"http://code.activestate.com/recipes/langs/python/tags/meta:requires=urllib2/2016-04-28T13:21:43-07:00ActiveState Code RecipesHow to use Python to convert a web page to PDF with a POST request to SelectPdf Online API and save it on the disk (Python)
2015-11-16T14:52:17-08:00SelectPdfhttp://code.activestate.com/recipes/users/4193129/http://code.activestate.com/recipes/579126-how-to-use-python-to-convert-a-web-page-to-pdf-wit/
<p style="color: grey">
Python
recipe 579126
by <a href="/recipes/users/4193129/">SelectPdf</a>
(<a href="/recipes/tags/api/">api</a>, <a href="/recipes/tags/converter/">converter</a>, <a href="/recipes/tags/htmltopdf/">htmltopdf</a>, <a href="/recipes/tags/pdf/">pdf</a>, <a href="/recipes/tags/selectpdf/">selectpdf</a>).
</p>
<p>This code converts an url to pdf in Python using SelectPdf HTML To PDF REST API through a POST request. The parameters are JSON encoded. The content is saved into a file on the disk.</p>
Python script to login to a website and convert required html page to pdf (Python)
2015-07-29T18:28:25-07:00Emil george jameshttp://code.activestate.com/recipes/users/4191910/http://code.activestate.com/recipes/579081-python-script-to-login-to-a-website-and-convert-re/
<p style="color: grey">
Python
recipe 579081
by <a href="/recipes/users/4191910/">Emil george james</a>
(<a href="/recipes/tags/bs4/">bs4</a>, <a href="/recipes/tags/pyhon/">pyhon</a>, <a href="/recipes/tags/pyqt4/">pyqt4</a>, <a href="/recipes/tags/python_scripts/">python_scripts</a>, <a href="/recipes/tags/urllib2/">urllib2</a>).
</p>
<p>This login script will allow you to login a website with python. After logging in, You now have access to all the pages for “members only” accessed with python and it can convert the required pages to pdf format save automatically in your /home/.</p>
<p>In this script i use anna university official website to get the exam result after that it convert the result page into pdf format and save to /home/.</p>
<p>Modules used :</p>
<ol>
<li><p>urllib2</p></li>
<li><p>BeautifulSoup(bs4)</p></li>
<li><p>requests</p></li>
<li><p>sys</p></li>
<li><p>PyQt4</p></li>
</ol>
<p><a href="https://emilgeorgejames.wordpress.com/2015/07/15/python-script-to-login-to-a-website-and-convert-required-html-page-to-pdf/" rel="nofollow">https://emilgeorgejames.wordpress.com/2015/07/15/python-script-to-login-to-a-website-and-convert-required-html-page-to-pdf/</a></p>
simplify webpage (Python)
2015-11-18T14:02:19-08:00Anton Vredegoorhttp://code.activestate.com/recipes/users/2667360/http://code.activestate.com/recipes/579107-simplify-webpage/
<p style="color: grey">
Python
recipe 579107
by <a href="/recipes/users/2667360/">Anton Vredegoor</a>
(<a href="/recipes/tags/adblocking/">adblocking</a>, <a href="/recipes/tags/drag_and_drop/">drag_and_drop</a>, <a href="/recipes/tags/html2text/">html2text</a>, <a href="/recipes/tags/tkinter/">tkinter</a>, <a href="/recipes/tags/tor/">tor</a>).
Revision 4.
</p>
<p>drag and drop urls from your browser's navigation window to a tkinter widget</p>
Python script to find linux distros details from distrowatch (Python)
2015-07-29T18:24:23-07:00Emil george jameshttp://code.activestate.com/recipes/users/4191910/http://code.activestate.com/recipes/579038-python-script-to-find-linux-distros-details-from-d/
<p style="color: grey">
Python
recipe 579038
by <a href="/recipes/users/4191910/">Emil george james</a>
(<a href="/recipes/tags/beautifulsoup/">beautifulsoup</a>, <a href="/recipes/tags/internet/">internet</a>, <a href="/recipes/tags/module/">module</a>, <a href="/recipes/tags/python/">python</a>, <a href="/recipes/tags/url/">url</a>, <a href="/recipes/tags/web/">web</a>).
</p>
<p>this script is a simlpe python script to find linux distros details from distrowatch using beautifulsoup,urllib2 modules.The script finds distros distribution details from <a href="http://distrowatch.com" rel="nofollow">distrowatch.com</a> when the distribution name is called as argument.</p>
Python Awesome DD (Python)
2014-07-12T07:29:59-07:00Mike 'Fuzzy' Partinhttp://code.activestate.com/recipes/users/4179778/http://code.activestate.com/recipes/578907-python-awesome-dd/
<p style="color: grey">
Python
recipe 578907
by <a href="/recipes/users/4179778/">Mike 'Fuzzy' Partin</a>
(<a href="/recipes/tags/curses/">curses</a>, <a href="/recipes/tags/dd/">dd</a>, <a href="/recipes/tags/file/">file</a>, <a href="/recipes/tags/input/">input</a>, <a href="/recipes/tags/output/">output</a>, <a href="/recipes/tags/progress_bar/">progress_bar</a>).
Revision 3.
</p>
<p>Imagine if the *nix utilities 'dd', 'wget', and 'pv' had a baby. That baby would be named 'padd'. A 'dd' replacement (most commonly used options translate with no problem), that gives progress output like 'pv' and supports (currently) http(s) and ftp input sources, with more planned (sftp for one). Output support for ftp, sftp, and http(s) is also planned.</p>
check_svn_number_for_all_modules (Python)
2014-04-14T07:41:43-07:00Shane Wanghttp://code.activestate.com/recipes/users/4171830/http://code.activestate.com/recipes/578861-check_svn_number_for_all_modules/
<p style="color: grey">
Python
recipe 578861
by <a href="/recipes/users/4171830/">Shane Wang</a>
.
</p>
<p>this tool is used to check whether an SVM number is included in current build or not.
please note that this is only for internal use.</p>
Music Downloader (Python)
2013-05-25T06:52:51-07:00Christian Careagahttp://code.activestate.com/recipes/users/4186639/http://code.activestate.com/recipes/578530-music-downloader/
<p style="color: grey">
Python
recipe 578530
by <a href="/recipes/users/4186639/">Christian Careaga</a>
(<a href="/recipes/tags/download/">download</a>, <a href="/recipes/tags/downloader/">downloader</a>, <a href="/recipes/tags/music/">music</a>, <a href="/recipes/tags/program/">program</a>, <a href="/recipes/tags/python/">python</a>, <a href="/recipes/tags/python_scripts/">python_scripts</a>, <a href="/recipes/tags/selenium/">selenium</a>, <a href="/recipes/tags/urllib/">urllib</a>, <a href="/recipes/tags/urllib2/">urllib2</a>).
</p>
<p>A Python Program i wrote that downloads music from the web</p>
Multithreading Downloader Class (Python)
2012-07-22T07:44:20-07:00Itay Brandeshttp://code.activestate.com/recipes/users/4182927/http://code.activestate.com/recipes/578220-multithreading-downloader-class/
<p style="color: grey">
Python
recipe 578220
by <a href="/recipes/users/4182927/">Itay Brandes</a>
(<a href="/recipes/tags/downloader/">downloader</a>, <a href="/recipes/tags/multithread/">multithread</a>, <a href="/recipes/tags/multithreading/">multithreading</a>, <a href="/recipes/tags/urllib/">urllib</a>, <a href="/recipes/tags/urllib2/">urllib2</a>, <a href="/recipes/tags/urlopen/">urlopen</a>).
</p>
<p>Garbs files from the web using multithreading in an attempt to enhance transfer rate.</p>
Set windows 7 to lock itself (upon timeout) if no internet connection found - security measure (Python)
2012-07-09T20:56:22-07:00commentator8http://code.activestate.com/recipes/users/4182761/http://code.activestate.com/recipes/578200-set-windows-7-to-lock-itself-upon-timeout-if-no-in/
<p style="color: grey">
Python
recipe 578200
by <a href="/recipes/users/4182761/">commentator8</a>
(<a href="/recipes/tags/lock/">lock</a>, <a href="/recipes/tags/password/">password</a>, <a href="/recipes/tags/python/">python</a>, <a href="/recipes/tags/windows/">windows</a>, <a href="/recipes/tags/windows_registry/">windows_registry</a>).
Revision 2.
</p>
<p>This script (or exe if using web2py with """setup(windows=['locker2.py']))""") can be run as a task in windows every x minutes and will test for the presence of an internet connection and depending on whether it is found will set windows to lock after a given timeout without user activity.</p>
<p>This was made with help from random code snippets from around the web.</p>
<p>Tested only on windows 7.</p>
Geocoding Lists via Google Maps (Python)
2012-05-11T05:06:27-07:00Mano Bastardohttp://code.activestate.com/recipes/users/4182040/http://code.activestate.com/recipes/578126-geocoding-lists-via-google-maps/
<p style="color: grey">
Python
recipe 578126
by <a href="/recipes/users/4182040/">Mano Bastardo</a>
(<a href="/recipes/tags/batch/">batch</a>, <a href="/recipes/tags/coordinates/">coordinates</a>, <a href="/recipes/tags/geocode/">geocode</a>, <a href="/recipes/tags/geocoding/">geocoding</a>, <a href="/recipes/tags/google/">google</a>, <a href="/recipes/tags/google_maps/">google_maps</a>, <a href="/recipes/tags/lat/">lat</a>, <a href="/recipes/tags/latitude/">latitude</a>, <a href="/recipes/tags/list/">list</a>, <a href="/recipes/tags/list_comprehension/">list_comprehension</a>, <a href="/recipes/tags/lng/">lng</a>, <a href="/recipes/tags/longitude/">longitude</a>, <a href="/recipes/tags/map/">map</a>, <a href="/recipes/tags/web/">web</a>).
Revision 2.
</p>
<p>A simple script written as an experiment in geocoding addresses in a database. A list of addresses in the form of "100 Any Street, Anytown, CA, 10010" is passed to a Google Maps URL, and the latitude/longitude coordinates are extracted from the returned XML.</p>
<p>XML methods are not used in this script, but simple string searches instead.</p>
Dropbox file uploader via web interface using Python with urllib2 and mechanize (Python)
2016-04-28T13:21:43-07:00ccpizzahttp://code.activestate.com/recipes/users/4170754/http://code.activestate.com/recipes/578030-dropbox-file-uploader-via-web-interface-using-pyth/
<p style="color: grey">
Python
recipe 578030
by <a href="/recipes/users/4170754/">ccpizza</a>
(<a href="/recipes/tags/dropbox/">dropbox</a>, <a href="/recipes/tags/mechanize/">mechanize</a>).
Revision 7.
</p>
<p>UPDATE:
This is script is not maintained and does not anymore with the current version of Dropbox. For a proper command line interface to dropbox I recommend <code>dropbox_uploader</code>: <a href="https://github.com/andreafabrizi/Dropbox-Uploader" rel="nofollow">https://github.com/andreafabrizi/Dropbox-Uploader</a></p>
<p>Originally inspired by the example at <a href="http://joncraton.org/blog/62/uploading-dropbox-python" rel="nofollow">http://joncraton.org/blog/62/uploading-dropbox-python</a>. </p>
<p>The script uses mechanize to logon to the web page and upload the file(s) to the Dropbox root folder or to the folder supplied on the command line as <code>dir:/my_dropbox_path</code> (if present, this must be the first parameter).</p>
<p>Multiple files and/or glob patterns names are accepted as script arguments.</p>
<h5 id="example-usage">Example usage</h5>
<pre class="prettyprint"><code>dropbox.py file1.txt # upload to root folder
dropbox.py dir:/Backups/2012 file1.txt # upload to custom folder
dropbox.py dir:/Backups/2012 *.txt # upload by file mask
dropbox.py dir:/Backups/2020 * # upload all files in current dir
</code></pre>
<p>Limitations: only files in current folder are processed, subfolders are ignored.</p>
<h5 id="note">NOTE</h5>
<p>The script requires the <code>mechanize</code> module - use <code>pip install mechanize</code> or <code>easy_install mechanize</code> to add it to your site-packages.</p>
<h5 id="note2">NOTE2</h5>
<p>I have found a cleaner way to manage dropbox files from the console - see the <em>dropbox-uploade</em>r script at <a href="https://github.com/andreafabrizi/Dropbox-Uploader" rel="nofollow">https://github.com/andreafabrizi/Dropbox-Uploader</a> - it is a Bash script that works using the official Dropbox API rather than emulating a web browser.</p>
ActiveState recipe importer (Python)
2011-11-23T02:27:51-08:00nosklohttp://code.activestate.com/recipes/users/4166478/http://code.activestate.com/recipes/577958-activestate-recipe-importer/
<p style="color: grey">
Python
recipe 577958
by <a href="/recipes/users/4166478/">nosklo</a>
(<a href="/recipes/tags/activestate/">activestate</a>, <a href="/recipes/tags/code/">code</a>, <a href="/recipes/tags/import/">import</a>, <a href="/recipes/tags/library/">library</a>).
</p>
<p>Finally! This code allows you to import any activestate recipe right into your code!</p>
<p>Example:</p>
<pre class="prettyprint"><code>>>> from activestate.recipe194373 import mreplace
>>> print mreplace('ectave steta racipas rock!', ('a', 'e'), ('e', 'a'))
active state recipes rock!
</code></pre>
<p>Save this as <strong>activestate.py</strong></p>
Youtap - Downloading YouTube video through Python command line script (Python)
2011-07-26T21:47:24-07:00Sundar Srinivasanhttp://code.activestate.com/recipes/users/4177884/http://code.activestate.com/recipes/577807-youtap-downloading-youtube-video-through-python-co/
<p style="color: grey">
Python
recipe 577807
by <a href="/recipes/users/4177884/">Sundar Srinivasan</a>
(<a href="/recipes/tags/flash/">flash</a>, <a href="/recipes/tags/mime/">mime</a>, <a href="/recipes/tags/youtube/">youtube</a>, <a href="/recipes/tags/youtube_downloader/">youtube_downloader</a>).
Revision 3.
</p>
<p>Python program to download YouTube video from command line. Originally posted in: <a href="https://github.com/krishnasun82/youtap" rel="nofollow">https://github.com/krishnasun82/youtap</a></p>
<p>Usage: python youtap.py "<youtube-link>"</p>
<p>The reason for giving the link in double-quotes is that sometimes the link contain '&'(ampersand), which the UNIX interprets as "run the program in background"</p>
Download all lolcat images from iCanHasCheezburger.com (Python)
2011-03-10T08:49:14-08:00Rahul Anandhttp://code.activestate.com/recipes/users/4173646/http://code.activestate.com/recipes/577603-download-all-lolcat-images-from-icanhascheezburger/
<p style="color: grey">
Python
recipe 577603
by <a href="/recipes/users/4173646/">Rahul Anand</a>
(<a href="/recipes/tags/download/">download</a>, <a href="/recipes/tags/images/">images</a>, <a href="/recipes/tags/lolcat/">lolcat</a>, <a href="/recipes/tags/python/">python</a>, <a href="/recipes/tags/web/">web</a>).
</p>
<p>Running this python script will download all lolcat images from <a href="http://icanhascheezburger.com" rel="nofollow">icanhascheezburger.com</a> to the current folder. Download will start from the oldest image. Images are collected into subfolders lolcat0, lolcat1 etc, each containing 300 images. The script can be stopped and resumed at anytime.
Make sure to create files <em>lolconfig.txt</em> and <em>log.txt</em> in the same folder before running the script. <em>lolconfig.txt</em> must have a string as follows in the beginning: <em>1496/1496/0</em>.
log.txt is an empty file in the beginning</p>
Wiki Recent Changes Checker (Python)
2010-12-28T07:27:25-08:00Ryhttp://code.activestate.com/recipes/users/4176376/http://code.activestate.com/recipes/577522-wiki-recent-changes-checker/
<p style="color: grey">
Python
recipe 577522
by <a href="/recipes/users/4176376/">Ry</a>
(<a href="/recipes/tags/beautifulsoup/">beautifulsoup</a>, <a href="/recipes/tags/python/">python</a>, <a href="/recipes/tags/ubuntu/">ubuntu</a>, <a href="/recipes/tags/wiki/">wiki</a>).
</p>
<p>Basic script that will check and see if there have been any recent changes to a Wikimedia. Used for low use Wikimedia sites to keep an eye on them. I run this whenever my box starts and it will open the browser along with a few other windows that open.</p>
Robot Pager (Search engines and others) (Python)
2010-10-10T19:33:20-07:00Carlos del Ojohttp://code.activestate.com/recipes/users/4173977/http://code.activestate.com/recipes/577420-robot-pager-search-engines-and-others/
<p style="color: grey">
Python
recipe 577420
by <a href="/recipes/users/4173977/">Carlos del Ojo</a>
(<a href="/recipes/tags/automate/">automate</a>, <a href="/recipes/tags/engine/">engine</a>, <a href="/recipes/tags/paging/">paging</a>, <a href="/recipes/tags/robot/">robot</a>, <a href="/recipes/tags/search/">search</a>, <a href="/recipes/tags/websites/">websites</a>).
Revision 3.
</p>
<p>This is a class to make easy the development of robots, to parse results over a website with a paging. For example Google, Yahoo, Bing, or any other page with paging system.</p>
<p>PagerEngine is the main class. I've developed three more clases implementing GoogleSearch, YahooSearch and BingSearch as examples.</p>
<p>Inheriting from PagerEngine (and having RexExp knowledge) you can easily develop other robots for other websites.</p>
Improved ReportLab recipe for "page x of y" (Python)
2009-07-06T10:03:28-07:00Vinay Sajiphttp://code.activestate.com/recipes/users/4034162/http://code.activestate.com/recipes/576832-improved-reportlab-recipe-for-page-x-of-y/
<p style="color: grey">
Python
recipe 576832
by <a href="/recipes/users/4034162/">Vinay Sajip</a>
(<a href="/recipes/tags/pdf/">pdf</a>, <a href="/recipes/tags/reportlab/">reportlab</a>).
Revision 2.
</p>
<p>This recipe is based on <a href="http://code.activestate.com/recipes/546511/"><a href="http://code.activestate.com/recipes/546511/">Recipe 546511</a></a> which does not work reliably if there are images in the content.</p>
Routines for programmatically authenticating with the Google Accounts system at Google App-Engine. (Python)
2010-05-20T20:39:50-07:00Berendhttp://code.activestate.com/recipes/users/4173891/http://code.activestate.com/recipes/577217-routines-for-programmatically-authenticating-with-/
<p style="color: grey">
Python
recipe 577217
by <a href="/recipes/users/4173891/">Berend</a>
(<a href="/recipes/tags/auth/">auth</a>, <a href="/recipes/tags/authentication/">authentication</a>, <a href="/recipes/tags/gae/">gae</a>, <a href="/recipes/tags/google/">google</a>, <a href="/recipes/tags/http/">http</a>, <a href="/recipes/tags/sessions/">sessions</a>).
Revision 2.
</p>
<p>This takes two calls, one to the ClientLogin service of Google Accounts,
and then a second to the login frontend of App Engine.</p>
<p>User credentials are provided to the first, which responds with a token.
Passing that token to the _ah/login GAE endpoint then gives the cookie that can
be used to make further authenticated requests.</p>
Download chromium browser nightly builds for any OS (with proxy support) (Python)
2014-07-05T18:47:47-07:00ccpizzahttp://code.activestate.com/recipes/users/4170754/http://code.activestate.com/recipes/577162-download-chromium-browser-nightly-builds-for-any-o/
<p style="color: grey">
Python
recipe 577162
by <a href="/recipes/users/4170754/">ccpizza</a>
(<a href="/recipes/tags/chrome/">chrome</a>, <a href="/recipes/tags/chromium/">chromium</a>, <a href="/recipes/tags/download/">download</a>, <a href="/recipes/tags/unzip/">unzip</a>).
Revision 19.
</p>
<p>Downloads the latest Chromium browser build from <a href="http://commondatastorage.googleapis.com/chromium-browser-continuous/" rel="nofollow">http://commondatastorage.googleapis.com/chromium-browser-continuous/</a> using urllib2 or wget (with Python versions below 2.5) and unzips the downloaded zip file to a predefined folder.</p>
<p>To use a custom proxy define the <code>HTTP_PROXY</code> system variable.</p>
<p>The script will figure out the OS but you can also pass the platform as the first parameter (one of <code>win32, linux, linux64, mac</code>).</p>
<p><em>Prerequisites (!! only for Python versions below 2.5):</em></p>
<ul>
<li><p><code>wget</code> - usually already installed on most linux systems. Windows users can get it <a href="http://gnuwin32.sourceforge.net/packages/wget.htm">here</a>.</p></li>
<li><p><code>unzip</code> - used for unpacking the archive; usually already installed on most linux systems. Windows users can get it <a href="http://gnuwin32.sourceforge.net/packages/unzip.htm">here</a>.</p></li>
</ul>
<p>Both <code>wget</code> and <code>unzip</code> should be available in PATH (for Python 2.5+ native Python modules are used).</p>
<p>For most Linux distros this script does not make much sense since the built-in package managers do a better job of managing chromium builds and dependencies, but it still can be useful if you are using a stable Chromium build but would like to be able to test the nightly builds too.</p>
<p>For OSX an additional installation step will be performed using the <code>install.sh</code> that is included in the OSX build. The OSX installer will copy the package to <code>~/Applications/Chromium</code>, and set some permissions that are required for Chromium to run. Running the unpacked zip without doing the installation will most likely will not work because of missing executable permissions on some files.</p>
Retrieve Dell Warranty Information for all machines in AD Domain (Python)
2010-02-18T15:51:28-08:00Kenneth Keiterhttp://code.activestate.com/recipes/users/4173089/http://code.activestate.com/recipes/577056-retrieve-dell-warranty-information-for-all-machine/
<p style="color: grey">
Python
recipe 577056
by <a href="/recipes/users/4173089/">Kenneth Keiter</a>
(<a href="/recipes/tags/active_directory_scripts/">active_directory_scripts</a>, <a href="/recipes/tags/ad/">ad</a>, <a href="/recipes/tags/dell/">dell</a>, <a href="/recipes/tags/domain/">domain</a>, <a href="/recipes/tags/network/">network</a>, <a href="/recipes/tags/warranty/">warranty</a>, <a href="/recipes/tags/windows/">windows</a>).
</p>
<p>This snippet retrieves warranty information for all Dell machines in a domain and outputs a CSV of the results. </p>
<p>Should be run on a machine joined to an active directory or NT4 domain. May need to be updated to parse Dell's website if they update it (since they have no service tag API).</p>