Issues with cached variable types and recalculating workbook

Post your problems here. This forum is monitored by our tech support team.

Moderator: giles

Issues with cached variable types and recalculating workbook

Postby gbrayut on Sun Feb 15, 2009 10:54 am

I started using cached worksheets and ran into a bit of a bug due to having cached data but recalculating the class definitions each time. My code started acting strange: it would work once and then stop working after recalculating the worksheet. After clearing the cached data (which included cells containing instances of custom python classes) it would work again once, and then stop working.

The problem came down to using the isinstance(aCell.Value, MyClass) python method to determine what type of variable is being stored in a cached cell. It works the first time, but after the definition for MyClass gets recalculated, IronPython no longer sees the cached object as the same type as the new class and always returns false. This is frustrating, since printing the type of the class will show <class '__main__.MyClass'> for both the cached variable and for calling type(MyClass()) (the repr string on the type is exactly the same too), but comparing them using isinstance or "type(aCachedVar) == type(MyClass())" will always return false.

After finding the root of the problem, I checked to see if it was specific to Resolver One, and it turns out that the same thing happens when using IronPython 1.1, 2.0 or standard Python 2.5. Basically if you create an instance of an object, reload the definition for the class (using an exefile call or by typing the definition again in a console window), and then try to compare using the new class definition it will fail. It is a subtle example where "duck typing" fails, in that if you reload the definition for the type it creates a new kind of duck such that "type(aDuck) != type(duck())" :-(

Everything else works great (it still acts like a normal object, just with a bit of an identity crisis), and I was able to bypass the bug by using "str(type(aCachedVar)) == str(type(MyClass()))" but it took a good hour or two to figure out what was going on. I'm not sure if there is a way to cache the class definitions or create a different type of class that would not have this issue (probably would work fine if the class was imported from a C# assembly, but I haven't tried that yet) or if there is a way to only load the class definition once (ie... cache the class as well as the data).

It is definitely something to watch out for though when storing data from custom python classes into cached worksheets.
gbrayut
 
Posts: 35
Joined: Fri Dec 26, 2008 10:28 pm
Location: Salt Lake City, UT

Re: Issues with cached variable types and recalculating workbook

Postby michael on Sun Feb 15, 2009 12:25 pm

This is part of the way the Python works. You can redefine a new class with the same name as an existing one - and it will be a different class. Instances of the old class will still exist of course and *not* be instances of the new class.

Class definitions in user code get re-executed on every calculation, with the effect you have noticed - cached instances of the class are no longer instances of the new class.

The easiest way round this is to define the class in an external module and import it in user code. This way the class definition is only executed on the first import.

If you need to modify the class, then you can press shift-F9 to force a recalc that also clears out the import. This way your module is re-imported and the modified class definition executed.

Alternatively you could do something clever like making the class definition conditional and storing the class in the cache or dynamically creating a module and putting it into sys.modules.

All the best,

Michael Foord
michael
 
Posts: 209
Joined: Thu Nov 01, 2007 11:11 pm
Location: London, UK

Re: Issues with cached variable types and recalculating workbook

Postby gbrayut on Sun Feb 15, 2009 8:30 pm

Thanks for the advice. I moved the class definition into a python module and imported it under the Pre-Constants section using "from CachedCodeModule import MyClass" and that seems to have fixed the problem. It makes sense now, just was counter intuitive for someone coming from a static type world.

Also, I can still see some situations where this will be a bit of a headache, mainly when the class needs to access the Workbook or CellRange objects. After playing around with Resolver One and python for a few weeks I have found that there are basically 4 different ways to write user code, and each of them have their own advantages and disadvantages:

1. IronPython user-code directly written in Resolver one usercode sections.

Advantages: Great for simple code, has full access to Workbook and CellRange objects. Workbook and code is contained in a single file.

Disadvantages: Lacking IDE features (code completion, syntax checking, testing/debugging features etc...), Code gets reloaded every time (can cause issues when using cached worksheets)

When to use: simple code sections for non-cached worksheets or when a full IDE is not needed.

2. IronPython user-code written in external py file and executed using execfile("my.py") in Resolver One user code sections.

Advantages: Code can be edited in full IDE. Code has full access to the Workbook and CellRange objects (ie... runs in same scope as workbook). Can create external test harness and use breakpoints for testing in IDE. When development is finished you can copy code from py files and paste into user code sections to ship worksheet as one file.

Disadvantages: Code gets reloaded every time. Full test harness can be difficult to setup because you have to import a lot of classes and recreate the entire workbook, which takes a long time (better to use a mock workbook class that implements only the features required for testing)

When to use: Any external code that needs to be run in the same scope as the workbook but does not need to be cached. Using one file for each of the code sections works very well so that code can be written and tested in external IDE.

3. IronPython user-code written in external py file and imported into Resolver one user code sections using "from <filename> import MyClass".

Advantages: Code can be edited in full IDE. Code gets cached in Resolver one. Same testing and debugging options as #2

Disadvantages: Code does not have full access to the Workbook and CellRange objects as it will run in it's own scope. To access the workbook you have to pass an instance of the post-constants workbook object as a value to a class constructor and then store it as a global variable. Even then you have to recreate the CellRanges and end up using a lot of globals and duplicate code. This method is not ideal when your code is accessing the Workbook and Cells directly, but work very well for class definitions that should be cached across recalculations.

When to use: Any external code that should be cached and does not need to access any of the Resolver One classes or libraries. Class definitions for cached data types should be loaded using this method so that the definition does not get re-loaded and cause problems with comparing variable types.

4. .NET code written in full IDE, compiled into assembly, and imported into Resolver One user code section using clr, assembly.LoadFile(dllFile) and an import statement

Advantages: can code in any .NET language or make use of existing class libraries and assemblies. May have some performance benefits in some situations due to compiler optimizations and static typing. Assembly will be cached in workbook.

Disadvantages: code cannot access any of the Resolver One classes or libraries and knows nothing about the Workbook or CellRanges. Code must be compiled into an assembly before it can be used in Resolver One workbook.

When to use: Best for integrating with existing external libraries.
gbrayut
 
Posts: 35
Joined: Fri Dec 26, 2008 10:28 pm
Location: Salt Lake City, UT

Re: Issues with cached variable types and recalculating workbook

Postby michael on Mon Feb 16, 2009 11:36 am

That's a great summary, thank you very much for taking the time to post it here.

We are working on improving the code editing features of Resolver One, but I think it will always be a balancing act as to which code belongs in its own module and which belongs directly in the user code sections.

Testing is another issue. Code that lives in its own module will be easier to test, although using RunWorkbook it should be possible to programmatically test whole spreadsheets. You could test using mock objects - or have your test harness import a Workbook from the Resolver One Library package and test with real objects (most of the spreadsheet objects are not expensive to create). If you do go down this route we would be very interested to hear how you get on.

All the best,

Michael Foord
michael
 
Posts: 209
Joined: Thu Nov 01, 2007 11:11 pm
Location: London, UK


Return to Help and support

Who is online

Users browsing this forum: No registered users and 1 guest

cron