Marshal.ReleaseComObject in .Net
I recently had the opportunity to spend a lot of time with .Net and COM. The architecture for the application is a COM C++ dll containing the business logic called through a c# .Net 1.1 dll to abstract away some of the complexity in a classic ASP web application for the UI. The issue started simply, a user loads everything up then logs out and the next user causes the web server to crash, sometimes with a db driver error.
The issue was reported in a way that could be it’s own blog post on how users can get a bug report wrong. I began investigating the db driver. However, all signs indicated it was the right one.
Some more research isolated the exact sequence of actions. At first, it seemed random. As described above the app has three layers; c++, c#, asp. In most of the app the middle c# layer is done in VB6. When you login and work and logout in the VB6 portions nothing happens. When you login and workand logout in the c# then go back to VB6, nothing happens. But when you login and work in c# and logout, the next user to login causes IIS to fail. This is odd, because if an error was in the c# code why did
jumping through VB6 code fix the error. It seems like the cause is c# or c++, but where?
So, the next step was to fire up the VC6 debugger, attach to inetinfo, and hope to catch an exception in c++. c# has much less code to check, but all of the code is suspect. c++ could be killing IIS, but since you’re logging out the user might not get an error. This took a lot of time and got into aspects of COM I hope never to see again. Exceptions were happening as expected, but the errors made no sense. There where three areas. Two, in user generated code that gets run all the time by lots of time. One in computer generated code that is not available to the programmer. I fixed one of the user code areas, the other was not changeable. Stack is of note too, because it gave some really odd readings. The debugger would stop on a line of code and if the variables were readable the address for the object might be x0. That’s the address of null and it’s not possible to have an object there much less have properties.
It seemed like there was a bug in our Standard Template Library (STL). Windows had a report for a fixed defect with somewhat similar properties. For the moment I set this idea aside. Changing STL is very expensive and the functions that errored are very basic. A real STL bug would have shown up years ago.
The error I was able to fix in the c++ code was a clue. I could track it back to a line of c# code. So, I tried modifying c# without changing the purpose of the code. The first attempt was changing a one line call to a two line and it was successsful. Like below. I could undo my c++ change and the 2 lines in c# would keep the error from occurring.
MyObject myObject = objSession.objectManager.GetNewMyObject();
to
ObjectManager objectManager = objSession.objectManager;
MyObject myObject = objectManager.GetNewMyObject();
This didn’t particularly make sense. I was happy it worked, but the advice I got was that it shouldn’t matter. However, we getting into the interface between Microsoft old COM and Microsoft new .Net. In the end I understood exactly why this worked.
After some research into COM/.Net, related memory leaks, attempts to contents of the garbage collector, and some well place debug messages, I firmly believed that the cause of my defect were unreleased parts of the c++ COM app in .Net’s garbage collector that leaves memory in a bad state the next time an application runs. In the above code, objSession is released, but objectManager or myObject is not. .Net tells Windows it still has a reference to those objects. When you run the VB6 part of the application the c# code is destroyed completely and the COM(c++) on COM(VB6) action works out the inconsistencies.
Now for the .Net wackiness. I found out to my surprise that you might have to tell .Net to release/destroy a COM object more times than you created it. For example, the function GetDependencyArray might create many more myObjects in memory. If you destroy only one the others may or may not cause a memory leak.
ObjectManager objectManager = objSession.objectManager;
MyObject myObject = objectManager.GetNewMyObject();
System.Array ar = myObject.GetDependencyArray();
If you dig a little deeper you’ll see that .Net releasing a COM object is not like anything else in another language. To destroy a COM object you release it, which signals to the Garbage Collector (GC) that it can reuse that memory when it wants to. You’re done with it. The call for the above example would be:
int count = Marshal.ReleaseCOMObject(myObject);
myObject = null;
The integer that is returned tells you how many more COM objects of the same rough type there are left in memory. Setting myObject to null keeps you from accidentally trying to use the object and get an error about how the RCW has already been released.
Since you may have multiple instances that you’re unaware of MS suggests running this in a loop till the count is 0. In .Net 2.0, they have a function to run the loop for you, FinalReleaseCOMObject.
do {
int count = Marshal.ReleaseCOMObject(myObject);
} while (count > 0);
myObject = null;
The wierd part is that ReleaseCOMObject only cares about the type of myObject. This is a problem, because it provides a global destruction spaces for all COM objects. Any part of the application can repeatedly call ReleaseCOMObject and destroy the COM objects used by the rest of the program that have that same type. In most languages you want the memory address of the object and release that. Once the object/address is gone it’s gone. You can’t do it twice and you certainly can’t destroy another object, just because they have the same type.
It’s hard for me not to get really pissed off and rant for the rest of the blog. In my opinion MS fucked this up and didn’t look back. Here’s why and the cause of my month long trek to defect resolution.
I read Sam Gentile’s blog, the .Net team blog, and a few programmer websites. They all said the same thing. Run ReleaseCOMObject in a loop, review your program to keep from wiping out too many objects, and next time don’t call COM from .Net it’s not designed to do that very well. You should use all .Net. After all anything else is unmanaged code.
My requirements were to look for every single line that created a COM object and provide a destructor. It was not unreasonable for two or more parts of the application to have a reference to the same object. And it’s impossible for the parts to know about the other’s reference so that they don’t release it causing a runtime error. When the .Net code closes all COM objects everywhere must be released. This solves my original defect. I never did figure out where in c# it was causing the c++ errors.
This is my solution and it’s unique so far as I’ve looked on the web. First, you need to record the reference count of every type of COM object that’s released. Any that aren’t 0 are suspect. Next, you need to create an object cache for the COM objects. Remember, you only need one cached object for each type of COM object. You can use Reflection to get the types. Store that and your reference count. I also store the variable name for debug tracking purposes. You need two functions, one Releases once, no loop. Most times only one call is needed. If the reference count is greater than 0, the object is saved in the cache. The other function is a FinalRelease. The name is misleading, because FinalRelease always leave one object in memory. When you’re program exits it’s time to release everything completely. Go through the object cache and release them till the count is 0. I also have it spit out a debug message when it is not 0. Sometimes, the object cacher releases everything and the normal destructor runs and drops the reference count to -1.
What would have made this a lot easy would be very easy for MS to have done in .Net 2.0. After all they had four years.
- Make the reference count a public variable.
We all know it’s there, but the only time you can see it is when you release a COM object. There is not reason to not provide and interface. Then you could record the count when your functions starts and release down to that count when your function stops. SIMPLE solution. Private to public, wow. - Provide an object cache mechanism like I just created.
I’m no genius, why didn’t MS figure this out. Certainly, a layer to deal with specifically with this type of COM/.Net interface is to be expected. What’s with the ability to release all objects globally. OMG, are you kidding me? What part of the first day of programming in the first class of CS did they not understand? - Interops should make destructors that call ReleaseCOMObject.
One of the solutions I really liked, but didn’t have time to implement was to provide a layer over the .Net Interop to the COM object. For each object you would add a destructor that calls ReleaseCOMObject. Now, you want to be careful and not release all COM objects, but with #1 and #2 that should be easier. Why am I required to do this? Can’t the Interop? After all it’s solely created to be the interface between .Net and COM. - Release COM Objects used by a .Net dll when .Net is destroyed.
The root problem wouldn’t have existed if .Net kept track, like I had to do, of which .Net app owned which COM object. When that .Net app is cleaned up so should all it’s COM references. Maybe technically difficult, but seems obvious that is should be done. What’s a Garbage Collector that doesn’t collect garbage? - Release a COM object by giving only the type.
At one point, I thought, “there’s no need for a cache I’ll just make a COM reference to what I know is there and release them all.” After all ReleaseCOMObject, doesn’t need a real COM Object. It just uses the type. So, why not let us pass in a type or type string and release those objects. MS, don’t abstract in complexity under the guise of simplicity. - ReleaseCOMObject sets the object to null
Why do I always have to have a second line to set the object to null after I released it? It’s dumb and makes my code verbose. Should the object ipsofacto be set to null since I just released. MS, why do you leave it open to cause an error when something else tries to call the released unnulled object?
MS knew .Net and COM would have to work together. After all COM has been around for a decade or more and by now and thousands, maybe millions of apps, large and small, are written using it. MS says you can call COM from .Net, but as this experience outlines, it’s not easy and is buggy. There should be a range of functions in .Net that tap into COM or are available to COM. Since, I haven’t written the COM that my .Net app is call I don’t know how I could write it better for .Net. Though you should be able to just like you had to tweak c++ for VB6.
My perspective is a VB6 programmer. It’s not complicated, you can write very quickly, programs are a little slow, but mostly it just works. I have no idea what managed or unmanaged code is and I don’t care. My job isn’t space shuttle software or real time nuclear reactor control. Just make it work.
Why is it this bad, still after 5-6 years? My guess is this interface deal was a hard task. And it was not given top priority, as it should. A bit of programmers would rather work on new and shiney than old and reliable. I’m sure it’s on the list, MS isn’t stupid nor do they hire stupid people. Based on experience, these issues will be resolved after the switch to .Net from COM has hit the tipping point. Another 5-6 years, at around .Net 5.
I’m just pissed I had to clean up their crap knowing they will fix it themselves eventually and more efficiently than I could.
Useful Links
Professional Trees & Shrubs II images
Selecting and Planting Landscape Trees
Amarillo Association of Realtors
Follow the links to the realtor’s sites to find the address of most homes
Patio Garden Pics
I’ve planted alot of herbs and vegetables this year. Part of it is in anticipation that some will die and part because it’s so easy. I research beneficial plants on the internet. If you place certain ones together they can help ward of each other bugs and diseases. The flowers are a cheap addition to help provide ground cover and reduce the loss of water from the container and cool the roots. The greenhouse is a box with the sides cut out and plastic wrap taped in. Here are the pics.
The list so far is (4/26/06):
10 patio tomatos (wow all the seeds sprouted)
1 grape tomato
3 cucumbers
1 catnip
1 sage
4 basil
1 fig
1 lavender
2 hops
3 okra
1 rosemary
Update 1 (5/25/06):
I’ve already had fruit off the grape tomato. There have been some additions, replacements, and deaths. Take a look. I’m impressed with how many tomatos have already produced and it’s still may. So, I purchased 2 more plants and I found lists of beneficial plants and planted them together. Overall, the patio should be pretty protected, it’s on the third floor, but I saw large grasshoppers up here last year.
The list so far is:
7-9 patio tomatos (some don’t seem to grow)
1 grape tomato (started bearing ~5/15/06)
2 cucumbers
1 catnip (replacement)
1 sage
4 basil (2 replacements, 1 blooming)
1 fig
1 lavender
2 hops (1 sprouted yesterday)
3 okra (1 almost dead, 1 sick, 1 healthy)
1 rosemary (unpurterbable)
2 german chamomile
1 citronella
2 goliath bush tomato (1 small pot, 1 half buried large pot)
1 ground cover peppermint\
1 sorrel
4 sweet marjoram (we’ll see who lives after the first wind storm)
1 snapdragon
Update 2 (6/25/2006):
My patio garden has been hit the past 3 weeks (that I was aware of) with some kind of infection agent either bacteria or fungus. It started on the big tomato plant and has progressed to everything except the sage, rosemary, basil, mesquito plant, mint, and fig. More root space seems to help the plant fight it off. It starts at the leaf ends. They brown and curl. As time goes on the curl becomes more pronounced and the stem is affected. New leaves don’t grow and may be defomed. Aphids seems to spread the problem. I’m treating the aphids by removing the bottoms of the self watering containers and spraying many times a day with soap and I have hung flypaper.
The disease is much more of a concern. I’m treating with a mixture of 1 qt water to 1 tsp baking soda, .5 tsp sesame oil, .5 tsp peanut oil, 3-4 drops regular dish soap. Some mishaps with neem concentrate removed all growth regions from the okra and stained the basil and snapdragon. The sage was completely unaffected by neem.
The fly traps pointed out how much dirt is blown out of the containers. So, I got some Impatiens and what was labeled a Sedum for ground cover. I’ve learned many things about ground cover. I hope to used the marjoram and/or peppermint as grass when I have a yard. Both are tough, stay low (now mowing), aggressive, and smell nice. The sedum joins this experiment. I think a melon or squash would be good in a garden as living mulch to cover the dirt of taller peppers, okra, tomatoes and such.
The list so far is:
~7 patio tomatos (too thick to know)
1 grape tomato (dying slowly)
2 cucumbers (fruit dies from leaf disease)
1 sage
4 basil (2 replacements, 4 blooming)
1 fig
1 lavender
2 hops (1 won’t grow and 1 is 4 ft on a trellis)
3 okra (all struggling)
1 rosemary (unpurterbable!)
2 german chamomile (leaf tips brown, they try to grow fast)
1 citronella
2 goliath bush tomato (1 small pot, 1 half buried large pot)
1 ground cover peppermint(growing quickly)
1 sorrel (struggling)
4 sweet marjoram (competes with peppermint, I’m amazed)
1 snapdragon (flowing again)
1 common ginger
1 potato
1 box of onions
6 Impatiens
2 sedums?
Update 3 (7/23/2006):
There’s been a lot of change the past few months. Two of the tomatoes, patio and goliath, have doubled in size and seem to be outgrowing the disease. The original plant to catch the disease is on it’s last legs, but still blooming and trying. The okra, which I thought was dead, must have had enough roots and has come back double the orignal size. The sorrel is barely making it. The citronella up and died in a two weeks. I have no idea why. The potato is almost dead in a week.
Some interesting notes. The aphids are impossible to wipe out. You can use gallons of soap, but that’s expensive. Fly paper works great, is very cheap, and bothers nothing. The disease seems to be controlled by the baking soda recipe on this site, using half peanut and half sesame as the oils. Be careful it will melt the petunias and severely hurt the okra. Most things seem unaffected. Perhaps the best thing I did was bring home a buck of half sand and half manure. That’s caused the tomatoes to get very big and slowed their disease quite a bit. The marjoram has been decimated, but the mint is unperterbable. The lettuce in the lettuce box doubled in size in a week. It looks very promising. When the weather is hot, water in the catch tray will actually be drawn up into the soil even though they are not in contact.
The list so far is:
~7 patio tomatos (too thick to know)
1 grape tomato (almost dead)
1 cucumbers (fruit dies from leaf disease)
1 sage(2-3 times bigger)
4 basil (2 replacements, 4 blooming)
1 fig(growing finally)
1 lavender
2 hops (1 won’t grow and 1 is up the trellis)
2 okra (doing well)
1 rosemary (unpurterbable!)
2 german chamomile (struggling)
2 goliath bush tomato (1 small pot no flowers, 1 half buried large pot fruiting well)
1 ground cover peppermint(growing quickly)
1 sorrel (struggling)
4 sweet marjoram (struggling)
1 snapdragon (flowing again)
1 common ginger(struggling)
1 potato(very wilted)
1 box of onions
1 box of black seeded simpson lettuce
6 Impatiens
2 sedums?
Update 4 (11/7/2006):
There’s been a lot going on in the patio garden since I last updated this post. A Pigeon family moved in, raised as son? and moved out. Lots of plants died. Lots of plants fruited. A couple of vegetables appeared out of nowhere (I didn’t plant them). Lots was learned.
The Pigeon is a whole other story. We’ll leave that for later. I’ll tell the story by pot. The potato died shortly after the last entry, in a week. I don’t know why, couldn’t get close enough to feel the soil because of the pigeons. The potato was pretty much gone when I got a Red Dragon stonecrop. I thought it could handle the inattention and sun pretty well. In fact it thrived (more later), never getting more than 4 inches tall, spreading, and not blooming. Light affects the leaves; more light more red, less light more green. The roots seem to run shallow, though digging found some 8 inches down they could have belong to the flax. I think I was actually overwatering the pot since I was unable to check it. When the baby bird was out there I through some flax seeds out for him to eat and they sprouted. After ~2 months they are 1ft tall slender towers of small leaves. The leaf disease may have affected some of the older leaves. I put a dark bronze flowing mum in there today from a 3″ pot.
There are two 30″ tomoto pots. One with group of patio tomatoes. One with a Goliath bush tomato. Both had the leaf disease pretty bad at times and almost non-existent at other times. Temperature was a factor. When the weather is 70-89 there’s little or no disease. Consistent temps below bring it out and temps about bring it out. Though below 60 it doesn’t do much, but neither did the plants. The spearmint did ok, about as good as the marjoram. The tom’s took a long time to develop fruit. Only in the last 6 weeks has there been much harvested. There are still many green tomatoes on the bushes, more than have been harvested. The petunias in the Goliath pot are a fraction of the size when I got them. I fertilized with some of the pigeon pop, nothing noticeable changed.
The hops are really struggling to stay alive, but the pots have been very entertaining. The trellises have given birds a chance to pop all in the pots and there are seeds. I discovered a 2 small tomatoes and a sunflower plant coming from nowhere. The red petunias did great. The leaf disease just burned up the leaves faster than the hops could out run them. Next year there should be dirt and more experience fighting the disease.
The Okra did pretty good till temps dipped below 50 at night. It wasn’t able to replace the leaves that the disease got. For about 6 weeks it was producing an Okra per week. The main stalk is about 3ft on one and 2.5ft on the other. The petunias did good, having the largest single plant and the largest impatiens. This pot held the sphagnum moss the best and after 4-5 months it’s impossible to tell any was there.
The sage is suffering from a desication of it’s leaves. The leaf disease? I trimmed the rosemary quite a bit. Both grew very well. The potted Goliath bush tomato in the small pot did much better than I thought for such a small pot. It was half destroyed by a pigeon landing. The grape tomato, first to contract the leaf disease, survived all year. I had counted it out many, many times. Towards the end it seemed able to outrun or manage the disease. The width was about 3.5-4 ft, thin and scraggly, but flowering and fruiting. Based on it I would say tomato size is somewhat dependent on pot depth. It’s pot was no wider than the Goliath plant, but twice as deep.
Ah, the Cucumber/Chamomile pot. They all died. The last to go was the Chamomile. I through some radish seeds and another kind of stonecrop in the pot to keep it from being empty. The radishes took a while to get on their feet, but I could harvest half a dozen right now. The stonecrop runs between everything. It’s made of runners that attach themselves to the soil.
Lastly, the large lettuce box bred fungus gnats like you wouldn’t believe. So many that they killed everything in it. I was so disgusted that I let it sit for 6 wks and hit is hard with the bug chemicals, forget organic. I replanted it and the gnats are returning, but that’s for the winter garden.
I seriously considered tossing all the dirt and plants this year that were contaminated, but that’s a hard investment to surrender and in dirt I won’t have the ability to toss diseased dirt and start over. So, I’ve been figuring out what affects it, what spreads it, and who isn’t infectable. Aphids? or fungus gnats spread it like crazy. Weather affects it, antifungal sprays too. Steer Manure was one of the greats elements I have under my control. It takes a week or two to become noticeable. Nothing on liquid fertilizer and the soap sprays are almost worthless. You have to drowned the gnats. A few mists directly on them does nothing.
A lot of stuff went on this year. Next year I expect to have a house with yard enough to plant more. Maybe a rabbit, something to help create an ecological system. I don’t know what to do with it all sometimes. There was more okra than I could eat and if the tomatoes hadn’t gotten sick dito on them.
The final list is:
5 patio tomatos
1 grape tomato (almost dead)
1 sage(2-3 times bigger)
4 basil
1 fig
2 hops
2 okra
1 rosemary (unpurterbable!)
2 german chamomile
2 goliath bush tomato (1 small pot, 1 half buried large pot)
1 ground cover peppermint
1 sorrel (struggling)
3 sweet marjoram
1 snapdragon
1 box of onions
1 box of black seeded simpson lettuce
2 Impatiens
12 sedums (red dragon and ?)