Importing a Python File by Name

[jefftk]

Link post

Let’s say you have a python script:

2024-01-24--evaluate-chimeras.py

And you want to pull a section of it out into a separate file:

2024-01-25--strand-split-artifacts.py

You would hope you could just do something like:

import ”/​path/​to/​2024-01-25--strand-split-artifacts.py”

But this doesn’t work: import wants a module name, not a filename. The simplest way I know to import a python file from a path is:

import sys
import importlib

sys.path.append(”/​path/​to/​”)
ssa = importlib.import_module(“2024-01-25--strand-split-artifacts”)

There are a lot of complicated ways to do this, some of which avoid needing to add something to sys.path, but for quick one-off research code better to keep it simple.

[Nathan Helm-Burger]

Ugh, this has caused me so many hours of pain over the years. Especially in the context of importing into jupyter notebooks, which can add a layer of complication.

[Nisan]

This article saved me some time just now. Thanks!

[Gunnar_Zarncke]

Why not use exec()?

[jefftk]

You mean with open("/path/to/foo.py") as inf: exec(inf.read())? I’d expect that to work, but:

• It doesn’t do namespacing (so instead of being foo.bar any symbols in foo will just end up as bar)

• I’d really prefer not to use exec: there are so many ways to misuse it, and it’s hard to tell if it’s being used properly from looking at it. Much less of an issue in research code, but I’d still rather not.

[Gunnar_Zarncke]

Yes, that’s what I meant. There are many ways to abuse exec, but your way of using importlib.import_module with tweaked path has the same security issues. One death you have to die and exec seems to be the simplest.

Maybe you explain a bit more why you want to pull out the section from the main file. What is the real purpose?

[jefftk]

I want to call the section from two different pieces of code I’m experimenting with right now. At some point some of it will likely end up in a real system, but right now it’s all very exploratory.

[Gunnar_Zarncke]

Then security doesn’t matter and you can do whatever is simplest. Even the module space doesn’t matter.