CLR Inside Out
IronPython
James Schementi
Contents
Dynamic Languages and Iterative Development
Line Noise
How the .NET Framework Fits In
IronPython Architecture
Putting IronPython to Work
Conclusion
IronPython is the code name for an implementation of the Python programming language written by the CLR team at Microsoft. IronPython runs on the Microsoft® .NET Framework and supports an interactive console with fully dynamic compilation. It is well integrated with the rest of the .NET Framework and makes all .NET libraries easily available to Python programmers, while maintaining full compatibility with the Python language.
This column will give a brief overview of Python and what sets dynamic languages apart from other languages. I will discuss iterative development, describe how IronPython integrates with .NET while staying true to Python syntax, and show the advantages of using IronPython to utilize .NET.
Dynamic Languages and Iterative Development
Dynamic programming languages allow for a program's structure to be changed while it runs: functions may be introduced or removed, new classes of objects may be created, and new modules may appear. These languages are usually dynamically typed, allowing variable declarations to change type during execution. But just having dynamic types does not make a language dynamic.
Python is a dynamic language with support for many programming paradigms such as object-oriented programming, structured programming, functional programming, and, with some extensions, aspect-oriented programming. While offering these choices, Python's designers are in favor of a sparse syntax to increase readability. It uses English keywords frequently where other languages use punctuation, and has notably fewer syntactic constructions than many structured languages. For example, Python uses indentation rather than curly braces to delimit blocks.
Here's an example of the Python function fib, which computes the Fibonacci numbers up to the specified value of n:
def fib(n): a, b = 0, 1 while b < n: print b # equivalent to temp = a; a = b; b = temp + b (aka swap pattern) a, b = b, a+b
The most common question I hear about dynamic languages is "why should I use them?" Trying to show a longtime C++ or C# developer that a dynamic language is useful for problems outside the scripting domain is difficult. One concrete benefit of dynamic languages is that they facilitate an iterative development cycle. The interactive capabilities of dynamic languages make them a better match for this type of cycle than compiled languages. Developers can focus on exploring and prototyping functionality, refactoring if necessary, and then repeating with new functionality. This allows for a more exploratory process of development than the code/build/run/debug loop that most other languages constrain the programmer to.
So how are dynamic languages able to achieve this? Most provide an interactive console, a high-level syntax, and extensive standard libraries.
The interactive console allows you to type in an expression and see it evaluated right away. Figure 1 shows the IronPython console in action.
Figure 1** Programming the IronPython Interactive Console **(Click the image for a larger view)
Line Noise
If you compare the amount of code that goes into simple C# or Python classes, you'll see that the concise syntax of dynamic languages makes it easier to write code more quickly. When creating a new class in a C# application, you have to switch from a creative mode into more of a bookkeeping mode, creating a new file and typing a chunk of boilerplate code before attacking what the class actually does. Extra syntax that does not provide any additional expressiveness to the program can be referred to as line noise. Python is much less "noisy."
The line noise in C# is there for a reason, whether for static typing or explicit declaration, but it can make the code difficult to read. Dynamic languages try to take the configuration responsibilities away from the programmer and put it on the compiler and interpreter. Dynamic typing is so popular in dynamic languages because the compiler/interpreter keeps track of types rather than requiring the programmer to explicitly state them. Figure 2 and Figure 3 show common programming tasks, swap and multi-value returns, with a line-noise comparison between C# and Python. Allowing the developer to write less code that accomplishes more is one of the pillars of dynamic languages.
Figure 2 Multiple Return Values in C# and Python
C#
object[] GetValues() { object[] results = new object[2]; object[0] = 42; object[1] = "Hello"; return results; } object[] results = GetValues(); int x = (int)results[0]; string s = (string)results[1];
Python
def get_values() : return 42, "Hello" x, s = get_values()
Figure 3 Swapping Two Variables in C# and Python
C#
void Swap(ref int x, ref int y) { int temp = x; x = y; y = temp; } int x = 4, y = 2; Swap(ref x, ref y);
Python
x, y = 4, 2 x, y = y, x
How the .NET Framework Fits In
Most dynamic languages ship with a large standard library. This "batteries included" philosophy means dynamic languages can be used for a broad range of tasks. These libraries usually include modules for writing Internet-facing applications, creating graphical user interfaces, connecting to relational databases, and even unit testing frameworks. Having these tools be part of the language, rather than an extra download module or other third-party library, allows you to write code faster. One of the drawbacks is that these standard libraries vary among languages. A common set of libraries among all languages would allow programmers to move from language to language easily.
The .NET Framework is intended to be a single runtime engine for many programming languages. It does this via a shared byte code intermediate language, just-in-time (JIT) and ahead-of-time compilers, a highly tuned garbage collector, and reflection and dynamic loading support. The academic world has envisioned this common runtime for decades, but .NET is the first runtime engine with many languages used in production today. Microsoft maintains .NET support for C#, Visual Basic®, C++, J#, and JScript®, while others have integrated languages like Eiffel, COBOL, FORTRAN, RPG, and Delphi with .NET.
The common language runtime (CLR) enables deep integration among languages; a single project could use many languages that work together seamlessly. This allows developers to choose the best language for the problems they are trying to solve, rather than settling on one language. A multi-language framework builds value from a larger developer base and allows the framework to provide a richer set of libraries to more languages.
This vision sounds a lot like the libraries each dynamic language provides, except that the libraries are different from language to language. The .NET Framework already supports many languages, but it could leverage the dynamic language space by providing common libraries for them.
IronPython Architecture
IronPython has come a long way very quickly. Our goals are to have dynamic language support for the CLR, finish IronPython 1.0 as a working example, and help other languages target the CLR. IronPython is released under the Microsoft shared source license, and releases are scheduled every few weeks until the version 1.0 release. The quick development cycle allows the team to quickly address customer needs.
IronPython uses a common type of compiler architecture; the noticeable difference is in code generation. The IronPython byte code is MSIL, intermediate language for the CLR, and will eventually be converted into native code (see Figure 4), whereas CPython (the standard Python implementation, done in the C language) has a runtime loop that interprets its byte code. The IronPython libraries are written in C# rather than C and called from the compiled IL, which is also how IronPython accesses .NET libraries. If you are interested in learning more about how IronPython works, take a look at the shared source release on the IronPython Web site.
Figure 4** IronPython Compilation Process **
IronPython has two important, often-conflicting goals: it needs to be seamlessly integrated with .NET and it needs to be a true implementation of the Python language. As a first-class .NET language, IronPython integrates simply and easily with standard .NET features. This includes subclassing classes and implementing interfaces from .NET. IronPython knows about common .NET libraries and namespaces like System, which can be imported and used in a straightforward manner.
>>> import System >>> from System.Collections import * >>> h = Hashtable() >>> h["a"] = "IronPython" >>> h["b"] = "Tutorial" >>> for e in h: print e.Key, ":", e.Value >>> ... a : IronPython b : Tutorial
To utilize additional .NET libraries, the clr module must be imported and then specific assemblies referenced. The additional library assemblies can then be imported and used seamlessly. Once the library is imported, it can be used just as before.
>>> import clr >>> clr.AddReference("System.Xml") >>> from System.Xml import * >>> d = XmlDocument()
IronPython needs to support all the dynamic features of Python, including adding existing fields, changing methods, and even changing types of instances at runtime. Also, the existing Python regression test and existing programs should run relatively unchanged, allowing today's Python users to use all the same scripts with IronPython.
This overlap of worlds introduces some interesting conflicts. IronPython handles them in ways that make both .NET and Python developers feel at home. Take the following code example:
>>> s = " py" >>> s.Trim() Traceback (most recent call last): File , line 0, in <stdin>##2 AttributeError: 'str' object has no attribute 'Trim' >>> import clr >>> s.Trim() 'py'
If you ask both sets of developers what the first two lines of code do, they will give you different answers. The .NET developer will say it removes the whitespace from the front of s. The Python developer will say it throws an exception since Trim is not defined on a Python string. Both are correct. The .NET developer should not have to learn a new library just to use the language and the Python developer needs code to work as expected. The solution is to default to the standard Python libraries until clr is imported. After this the Python types will turn into .NET types and act as such.
Exceptions are another language design challenge. For example, the statement 2/0 will produce different exceptions in .NET and Python. The IronPython solution is to automatically convert the exception to the one that the developer wants to catch. See the "Resources" sidebar for links to more information.
Putting IronPython to Work
Since IronPython has full use of the .NET Framework, it can utilize Windows® Forms to create GUI applications, use C# code and extend any existing C# code, or even be integrated into Visual Studio®! The following walkthroughs are adapted from the IronPython tutorial, which is located in the IronPython installation directory under Tutorial\Tutorial.htm; it is a fairly comprehensive overview, so running through the tutorial after reading this column is recommended.
The code to initialize IronPython for Windows Forms is provided as winforms.py. To learn why this code is needed, look at winforms.py or the Windows Forms section of the tutorial.
Begin by making sure your current directory is the Tutorial folder and starting the IronPython console by running ipy.bat. You should see the console appear. Next, initialize Windows Forms by loading the winforms module/script:
>>> import winforms
The Python modules are automatically initialized (executed) upon import so the Windows Forms initialization code has executed as part of the import statement.
Now you can import the contents of the System.Windows.Forms and System.Drawing namespaces into the global namespace:
>>> from System.Windows.Forms import * >>> from System.Drawing import *
Next, create an instance of the Form class and display it:
>>> f = Form() >>> f.Show()
Now set the form Text property:
>>> f.Text = "My First Interactive Application"
At this point you should see the window shown in Figure 5.
Figure 5** Windows Form Dialog **
Define a method to create a label at the position of a mouse click:
>>> def click(f, a): >>> l = Label(Text = "Hello") >>> l.Location = a.Location >>> f.Controls.Add(l)
Register the event handler:
>>> f.Click += click
Now clicking on the form with the mouse will add "Hello" labels as shown in Figure 6. You can also access the controls via mouse clicks and change them (see Figure 7):
>>> for i in f.Controls: i.Font = Font("Verdana", 15) ... >>> for i in f.Controls: i.Text = "World" ...
Figure 6** Fancy Labels **
Figure 7** Changing the Labels **
After a few moments of clicking, the form will get quite crowded, so clear it out:
>>> f.Controls.Clear()
Extending IronPython with C# is just as easy. This walkthrough will show you how to use C# to build a class that can be used from inside IronPython.
First, create a new file called csextend.cs and put the code shown in Figure 8 inside it. Open the Visual Studio 2005 Command Prompt and build the code using csx.bat. You can load the DLL you just built (csextend.dll) into IronPython. Then explore the Simple class using the built-in dir function, as shown here:
>>> import clr >>> clr.AddReferenceToFile("csextend.dll") >>> import Simple >>> dir(Simple) ['Equals', 'GetHashCode', 'GetType', 'ToString', '__new__', '__repr__']
Now create an instance of the Simple class:
>>> s = Simple(10) >>> print s Simple<10>
Figure 8 csextend.cs
using System; using System.Collections; public class Simple { private int data; public Simple(int data) { this.data = data; } public override string ToString() { return String.Format("Simple<{0}>", data); } }
The Visual Studio SDK ships with a Visual Studio Integration sample of IronPython. This features IronPython projects to be created, syntax coloring, debugging, and IntelliSense® in Python files and the IronPython console. This is another example of the benefits of being a full-fledged .NET citizen: Visual Studio support!
You can learn more about using IronPython in Visual Studio by visiting Aaron Marten's blog about Visual Studio Integration and following the direction at the post blogs.msdn.com/533273.aspx.
Resources
Conclusion
Dynamic languages are what make agile software development and iterative processes successful. Their exploratory nature makes them fun for developers, and people can enjoy them with just an interactive console.
IronPython has shown that the .NET Framework is a powerful platform for dynamic languages. IronPython's approach can be generalized, allowing languages to easily be made available on the .NET Framework and giving developers the flexibility of many languages with familiar libraries.
Now that you have a broad overview of what IronPython is, give this dynamic language a try (if you haven't already). The tutorial, source code, and Web site are valuable resources that will make your IronPython experience fun!
Send your questions and comments to clrinout@microsoft.com.
James Schementi is a Program Manager intern at Microsoft. He is working on the IronPython sample applications and Web stories. As a graduate student at Worcester Polytechnic Institute in Massachusetts, he focuses on language design and compilers, especially dynamic languages, Web design and usability, and Web frameworks. He can be reached at jimsch06@wpi.edu.