Welcome, guest | Sign In | My Account | Store | Cart

os.walk is a very nice replacement for os.path.walk, which I never did feel comfortable with. There's one very common pattern of usage, though, which still benefits from a simple helper-function; locating all files matching a given file-name pattern within a directory tree.

Python, 8 lines
1
2
3
4
5
6
7
8
import os, fnmatch

def locate(pattern, root=os.curdir):
    '''Locate all files matching supplied filename pattern in and below
    supplied root directory.'''
    for path, dirs, files in os.walk(os.path.abspath(root)):
        for filename in fnmatch.filter(files, pattern):
            yield os.path.join(path, filename)

An example - I needed to identify all malformed XML files within my project. With the help of locate (and ElementTree) all I needed was:

from xml.parsers.expat import ExpatError
import xml.etree.ElementTree as ElementTree

for xml in locate("*.xml"):
    try:
        ElementTree.parse(xml)
    except (SyntaxError, ExpatError):
        print xml, "\tBADLY FORMED!"

Thanks to Marius Gedminas and Tom Lynn for improvements incorporated in this code.

14 comments

Lucas Schifferle 17 years, 3 months ago  # | flag

a simple question. How could the script be modified so that it locates all files in the current (or specified) root directory only - not its sub-directories? Thanks.

Rafal Zawadzki 17 years, 3 months ago  # | flag

os.listdir(). import os

os.listdir()

Rafal Zawadzki 17 years, 3 months ago  # | flag

naturally i mean listdir as a keyword only.

import os, fnmatch

def locate(pattern, root=os.curdir):
    '''Locate all files matching supplied filename pattern in and below
    supplied root directory.'''
    #for path, dirs, files in os.walk(os.path.abspath(root)):
    files = os.listdir(os.path.abspath(root))
    for filename in fnmatch.filter(files, pattern):
        yield filename
Lucas Schifferle 17 years, 3 months ago  # | flag

a simple solution (?). Thanks for the help!

A solution to locate files matching a pattern inside the root dir only (so not its sub-dirs) could be:

def locate2(pattern, root=os.curdir):
    filename = fnmatch.filter(os.listdir(os.path.abspath(root)), pattern)
    for fn in filename:
        yield os.path.join(root, fn)
Lucas Schifferle 17 years, 3 months ago  # | flag

writing at the same time... Thanks again for the help!

Simon Brunning (author) 17 years, 2 months ago  # | flag

Locating files in a single directory. The glob module in Python's standard library will find files matching a pattern in a single directory. Check out the docs.

Martin Laloux 16 years ago  # | flag

glob example. import os, glob

dor = the path you want

for dir, subdir, files in os.walk(dor):

for file in files:


          if glob.fnmatch.fnmatch(file,"*.txt"):


               do what you want
Simon Brunning (author) 14 years, 1 month ago  # | flag

The glob module is much simpler than that:

for filename in glob.glob('*.txt'):
    print filename
Arup 11 years, 10 months ago  # | flag

Hi, I want to search folders which are having a general pattern like: pattern = "[0-9]+.[0-9]+.[0-9]+.[0-9]+" It means a folder name could be anything constitute of nos and four quads. example:7.0.0.1

But i unable to search this using above code,by defining the pattern as above mentioned by me. Help me please ASAP.

Simon Brunning (author) 11 years, 10 months ago  # | flag

@Arup What you have there is a regex pattern rather than a glob pattern, so you'll need methods from the re module rather than the fnmatch module. You'll need to adapt locate() accordingly.

Arup 11 years, 10 months ago  # | flag

@Simon, Thanks for pointing me this. Can you please help me to locate folders with pattern="[0-9]+.[0-9]+.[0-9]+.[0-9]+" using re module.

thiruvenkadam 11 years, 3 months ago  # | flag

Arup How about this?

for filename in glob.glob(...): if filename.replace(".", "").isdigit(): print filename

adbdkb 10 years, 3 months ago  # | flag

How can I use / modufy the above script to search in a way such that I can locate a file under directories with certain pattern under a root directory

What I mean is if I have a directory installedApps under which I have a.ear, b.ear,... x.ear, y.ear, z.ear directories along with other directories at the same level and i want to search for a file pattern web*.xml only under those subdirectories under the root without traversing other directories at the same level.

Can this be done?

Thank you

Simon Brunning (author) 10 years, 3 months ago  # | flag

@adbdkb

Something like this? (Untested.)

def locate(pattern, pathpattern="*", root=os.curdir):
    '''Locate all files matching supplied filename pattern in and below
    supplied root directory.'''
    for path, dirs, files in os.walk(os.path.abspath(root)):
        if fnmatch.fnmatch(path, pathpattern):
            for filename in fnmatch.filter(files, pattern):
                yield os.path.join(path, filename)
Created by Simon Brunning on Tue, 12 Dec 2006 (PSF)
Python recipes (4591)
Simon Brunning's recipes (5)

Required Modules

Other Information and Tasks