|
Source: ONLamp.com I haven’t read many RSS feeds lately due to my time constraints from writing a book, but I came across this post by Adam Gomaa, by way of some of the responses from Jonathan LaCour’s blog.
If there is one thing to learn from the endless, and pointless, comparisons of frameworks in Python, i.e, popularity contests, it is that most important things for a web frameworks are:
1. Documentation
2. Marketing
In a popularity contest, the “best” framework, is going to have the best documentation, and the best marketing, like tons of screencasts, etc. Every other discussion is an exercise in futility. If you want to be home coming queen, people have to know who you are, or they won’t vote for you. One of the reasons why Django is considered the most “popular” or “best” framework to many people, is that the Django people did an incredible job of documentation, and marketing.
Also, it is interesting, but not surprising that Zope 3, ZODB, Plone 3, and Grok are virtually non-existent in discussions about Python Web Development. I see a lot of discussions about people writing wiki’s and blogs, etc, in their discussions. I suppose I should probably bring up that fact that Plone 3 is one of the most kick*** Content Management Systems on planet earth, see PyATL. If you want a blog, install Plone, if you want a wiki, install MoinMoin, or Plone.
Finally, I think WSGI is going to change everything, and so some of the current comparisons are just silly, but it would great to live in a world in which the real issues are discussed like, “My documentation kicked your documentation’s *ss!”. He who wins the documentation and marketing, wins the hearts and minds!
Noah’s Personal Blog
Source: ONLamp.com Map files directly to memory using mmap.
Module: mmap
Purpose: Memory-map files instead of reading the contents directly.
Python Version: 2.1 and later
Description:
Use the mmap() function to create a memory-mapped file. There are differences in the arguments and behaviors for mmap() between Unix and Windows, which are not discussed below. For more details, refer to the library documentation.
The first argument is a fileno, either from the fileno() method of a file object or from os.open(). Since you have opened the file before calling mmap(), you are responsible for closing it.
The second argument to mmap() is a size in bytes for the portion of the file to map. If the value is 0, the entire file is mapped. You cannot create a zero-length mapping under Windows. If the size is larger than the current size of the file, the file is extended.
An optional keyword argument, access, is supported by both platforms. Use ACCESS_READ for read-only access, ACCESS_WRITE for write-through (assignments to the memory go directly to the file), or ACCESS_COPY for copy-on-write (assignments to memory are not written to the file).
File and String API:
Memory-mapped files can be treated as mutable strings or file-like objects, depending on your need. A mapped file supports the expected file API methods, such as close(), flush(), read(), readline(), seek(), tell(), and write(). It also supports the string API, with features such as slicing and methods like find().
Sample Data:
All of the examples use the text file lorem.txt, containing a bit of Lorem Ipsum. For reference, the text of the file is:
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
egestas, enim et consectetuer ullamcorper, lectus ligula rutrum leo, a
elementum elit tortor eu quam. Duis tincidunt nisi ut ante. Nulla
facilisi. Sed tristique eros eu libero. Pellentesque vel arcu. Vivamus
purus orci, iaculis ac, suscipit sit amet, pulvinar eu,
lacus. Praesent placerat tortor sed nisl. Nunc blandit diam egestas
dui. Pellentesque habitant morbi tristique senectus et netus et
malesuada fames ac turpis egestas. Aliquam viverra fringilla
leo. Nulla feugiat augue eleifend nulla. Vivamus mauris. Vivamus sed
mauris in nibh placerat egestas. Suspendisse potenti. Mauris massa. Ut
eget velit auctor tortor blandit sollicitudin. Suspendisse imperdiet
justo.
Reading:
To map a file for read-only access, make sure to pass access=mmap.ACCESS_READ:
import mmap
f = open('lorem.txt', 'r')
try:
m = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
try:
print 'First 10 bytes via read :', m.read(10)
print 'First 10 bytes via slice:', m[:10]
print '2nd 10 bytes via read :', m.read(10)
finally:
m.close()
finally:
f.close()
In this example, even though the call to read() advances the file pointer, the slice operation still gives us the same first 10 bytes because the file pointer is reset. The file pointer tracks the last access, so after using the slice operation to give us the first 10 bytes for the second time, calling read gives the next 10 bytes in the file.
$ python mmap_read.py
First 10 bytes via read : Lorem ipsu
First 10 bytes via slice: Lorem ipsu
2nd 10 bytes via read : m dolor si
Writing:
If you need to write to the memory mapped file, start by opening it for reading and appending (not with 'w', but 'r+') before mapping it. Then use any of the API method which change the data (write(), assignment to a slice, etc.).
Here’s an example using the default access mode of ACCESS_WRITE and assigning to a slice to modify part of a line in place:
import mmap
import shutil
# Copy the example file
shutil.copyfile('lorem.txt', 'lorem_copy.txt')
word = 'consectetuer'
reversed = word[::-1]
print 'Looking for :', word
print 'Replacing with :', reversed
f = open('lorem_copy.txt', 'r+')
try:
m = mmap.mmap(f.fileno(), 0)
try:
print 'Before:', m.readline().rstrip()
m.seek(0) # rewind
loc = m.find(word)
m[loc:loc+len(word)] = reversed
m.flush()
m.seek(0) # rewind
print 'After :', m.readline().rstrip()
finally:
m.close()
finally:
f.close()
As you can see here, the word shown in bold is replaced in the middle of the first line:
$ python mmap_write_slice.py
Looking for : consectetuer
Replacing with : reutetcesnoc
Before: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
After : Lorem ipsum dolor sit amet, reutetcesnoc adipiscing elit. Donec
ACCESS_COPY Mode:
Using the ACCESS_COPY mode does not write changes to the file on disk.
f = open('lorem_copy.txt', 'r+')
try:
m = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_COPY)
try:
print 'Memory Before:', m.readline().rstrip()
print 'File Before :', f.readline().rstrip()
print
m.seek(0) # rewind
loc = m.find(word)
m[loc:loc+len(word)] = reversed
m.seek(0) # rewind
print 'Memory After :', m.readline().rstrip()
f.seek(0)
print 'File After :', f.readline().rstrip()
finally:
m.close()
finally:
f.close()
Note, in this example, that it was necessary to rewind the file handle separately from the mmap handle.
$ python mmap_write_copy.py
Memory Before: Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
File Before : Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
Memory After : Lorem ipsum dolor sit amet, reutetcesnoc adipiscing elit. Donec
File After : Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
Regular Expressions:
Since a memory mapped file can act like a string, you can use it with other modules that operate on strings, such as regular expressions. This example finds all of the sentences with “nulla” in them.
import mmap
import re
pattern = re.compile(r'(.W+)?([^.]?nulla[^.]*?.)',
re.DOTALL | re.IGNORECASE | re.MULTILINE)
f = open('lorem.txt', 'r')
try:
m = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
try:
for match in pattern.findall(m):
print match[1].replace('n', ' ')
finally:
m.close()
finally:
f.close()
Since the pattern includes two groups, the return value from findall() is a sequence of tuples. The print statement pulls out the sentence match and replaces newlines with spaces so the result prints on a single line.
$ python mmap_regex.py
Nulla facilisi.
Nulla feugiat augue eleifend nulla.
References:
effbot.org - The mmap module
Python Module of the Week Home
Download Sample Code
Technorati Tags:
python, PyMOTW
|
