Various Gubbins.
By hulver (Tue May 04, 2004 at 09:33:58 PM EST) (all tags)
.net is more complicated than you think. Subtle problems that will bite you in the arse. I thought .net was supposed to fix things like this, not make them worse?

That .net thing really bites my bag. See, when I code something in Delphi, I know what it's doing. I understand that if I make something multithreaded, I have to deal with locking problems. I have to know that something might be calling a method while I'm running the destructor.

In .net, the object destructor might not even get run. The finalisation method is full of problems, and your objects might get resurected while you're destroying them.

What a fucked up over complicated system.

What ever happened it KISS?

In this case you're not even going to be able to KISS, because even if you do keep your code simple, a subtlety like this will jump up and bite you later.

Once again, MS is making things harder by trying to make them easier.

Bah.

Finalisation by spiralx (3.00 / 0) #1 Tue May 04, 2004 at 09:58:59 PM EST
I don't think I've ever used an object with a finalise() method in Java once. We do all of our destruction manually using a destroy() method. Works much better seeing as you can't guarantee when or if finalise() will be called in Java. Sounds like it's the same in .NET

Thing is by hulver (3.00 / 0) #3 Tue May 04, 2004 at 10:28:28 PM EST
If you're relying on the GC, there is no garuntee that the destroy method will be called at all.
--
Cheese is not a hat. - clock
[ Parent ]
Nope by spiralx (3.00 / 0) #4 Tue May 04, 2004 at 10:41:23 PM EST
Because the destroy() method is called manually when you're done with the object, followed by setting references to it to null. It's the finalise() method that's invoked by the GC.

[ Parent ]
multi stage construction/destruction is evil by codemonkey uk (6.00 / 2) #8 Tue May 04, 2004 at 11:43:51 PM EST
i is (as always) right. RAII is the only reasonable way to do these things. As soon as your classes sport "Initialize" or "Destroy" members you are entering a world of pain. It's more work for you to manage your object life times, and it's more work for you to verify your object invariants. You can't make any assumptions about your objects state, and end up littering every member with fail safe checks. It's all well and good saying that you'll be "setting references to it to null", but if your sure you can do that reliably, then why on earth are you using a GC language, and then plastering up your code to fix it's broken object model!?!

Almost as Smart As you.
[ Parent ]
Because by spiralx (5.00 / 1) #9 Tue May 04, 2004 at 11:56:52 PM EST
We're forced to do all this so it works with Microsoft's JVM implementation which hasn't changed in about five years now and is crap and fails to clear up anything until the browser is closed down. It was either this way or basically lose 5Mb every time you hit Refresh on your browser or go to another page. Sun's JVM is a lot better at clearing up after itself, but we've still got a lot of people using the MS one.

Yes, it's a pain in the arse and not ideal, but it works.

[ Parent ]
(Comment Deleted) by yicky yacky (3.00 / 0) #14 Wed May 05, 2004 at 01:11:35 AM EST

This comment has been deleted by yicky yacky

[ Parent ]
True by spiralx (5.00 / 1) #18 Wed May 05, 2004 at 01:19:25 AM EST
setting all references to null, calling gc(), or finalize(), has no effect on whether the clean-up is actually run or not - it's more like logging a request with the garbage collector to "prioritize this bit here when you have a moment".

This is true, there's never anything guaranteed. However by removing all references from all objects in the tree it does make a huge difference... as I said by intercepting the applet's stop() method and doing this we reclaim 5Mb of space that otherwise we'd lose, much of which was in arrays of data. And seeing as the JVM never gets rid of memory it's claimed, this was a serious problem.

[ Parent ]
(Comment Deleted) by yicky yacky (3.00 / 0) #19 Wed May 05, 2004 at 01:27:08 AM EST

This comment has been deleted by yicky yacky

[ Parent ]
Finalisation is useless. by i (6.00 / 2) #2 Tue May 04, 2004 at 10:03:37 PM EST
RAII all the way, baby.

Dammit, why do bad things happen to good- by Rogerborg (3.00 / 0) #5 Tue May 04, 2004 at 10:45:28 PM EST
No, wait, enterfornone?  All the children of the earth are perfect, but you shouldn't go chasing them.  It's sad that he's in trouble, but it was pretty obvious that he was a train wreck waiting to happen for some time.  I'll send him some love later.

-
Metus amatores matrum compescit, non clementia.

Why do you hate efn so? by Breaker (3.00 / 0) #16 Wed May 05, 2004 at 01:15:32 AM EST
Did he wee in your diary once?

[ Parent ]
not hate by iGrrrl (6.00 / 1) #20 Wed May 05, 2004 at 01:29:48 AM EST
But it's pretty easy to deduce from the way things were going that it was likely to implode.

So, from the other side of the planet, it's easy to say, "Saw that one coming."

"Beautiful wine, talking of scattered everythings"
(and thanks to Scrymarch)

[ Parent ]
What iGrrl said by Rogerborg (6.00 / 1) #32 Wed May 05, 2004 at 11:40:24 PM EST
I quite like him, but I always pictured him with an agitated twitchy look about him that would encourage you to sidle... slowly... away...

-
Metus amatores matrum compescit, non clementia.
[ Parent ]
Garbage collection is evil by Herring (6.00 / 4) #6 Tue May 04, 2004 at 10:49:28 PM EST
C++ all the way.

Bjarne Stroustrup he's our man
He can heal the sick with the touch of his hand

You can't inspire people with facts
- Small Gods

GC is not necessarily evil. by i (6.00 / 1) #12 Wed May 05, 2004 at 12:18:24 AM EST
But finalisation as we know it is.

[ Parent ]
I've yet to get my dirty little paws on Delphi 8 by TPD (3.00 / 0) #7 Tue May 04, 2004 at 11:40:55 PM EST
though it looks like I'll be doing a server port in a week or 3s time...

However I thought Delphi had in the main taken the decission to bypass the .NET finalizers?

this looks interesting but I haven't really studied yet...

http://bdn.borland.com/article/0,1410,29365,00.html.

why sit, when you can sit and swivel with The Ab-SwivellerTM

They have by hulver (5.33 / 3) #10 Wed May 05, 2004 at 12:02:38 AM EST
They implement it as an IDisposable interface. That article does explain a lot about the implementation.

It also goes on about the added complexity that's needed to almost correctly implement object destruction for objects that use unmanaged resources.

All this is going to do is introduce subtle bugs and race conditions that will be a nightmare to debug and will leave programmers scrabbling for the black arts in order to stop their programs crashing.

I already see this here. People developing in systems they have no control over end up implementing hacks to work around strange problems they don't understand.
--
Cheese is not a hat. - clock

[ Parent ]
I'll try to remember to sacrifice a cockrell by TPD (5.00 / 1) #11 Wed May 05, 2004 at 12:13:04 AM EST
before attempting the server port then!

why sit, when you can sit and swivel with The Ab-SwivellerTM
[ Parent ]
Ah! by Breaker (3.00 / 0) #17 Wed May 05, 2004 at 01:17:36 AM EST
People developing in systems they have no control over end up implementing hacks to work around strange problems they don't understand.

That's almost all of commercial programming though, neh?

[ Parent ]
What language are you using with .net? by i (3.00 / 0) #13 Wed May 05, 2004 at 12:43:22 AM EST
There's a neat resource management idiom in Java which might be applicable to you.
None yet, by hulver (3.00 / 0) #15 Wed May 05, 2004 at 01:13:20 AM EST
Thank Dog.
--
Cheese is not a hat. - clock
[ Parent ]
malloc + free by martingale (3.00 / 0) #21 Wed May 05, 2004 at 01:39:52 AM EST
This is slightly off topic, but still fits in. One of my utility programs written in C basically reads input, does a lot of hashing and parsing, and outputs a single number. It (can) eat a lot of memory if it reads say 50 megs of data, but basically it's written to do a few allocations and never touch memory until after it's all done.

What I'm wondering is if it makes sense to free() the memory myself or simply exit and let the kernel do the free'ing. What's generally faster on various OSes? The utility may well be called tens of thousands of times, one right after the other, each time taking a fraction of a second to run.

What I'm worried about is that when I run a shell script which calls up the utility repeatedly, calling time indicates that more than half the running time is system time, ie less than half the running time is user time. Other similar utilities, which can replace mine in the shell script, don't use up so much system time. How do I track down what takes up system time? [tests on Lunix 2.6.3].
--
$E(X_t|F_s) = X_s,\quad t > s$

Use a profiler. by i (3.00 / 0) #22 Wed May 05, 2004 at 01:58:40 AM EST
Which? I'd like to know myself :(

Or you might try to speed up the IO, by using mmap() and shit. Your own memory allocator is sometimes helpful too.

[ Parent ]
yeah, I played around with that by martingale (3.00 / 0) #25 Wed May 05, 2004 at 02:29:30 AM EST
It's quite weird. I'm using fopen(), but mmap() turns out to be no improvement at all. The utility takes a single pass through the data, and never backtracks or modifies the input. So maybe it's Lunix, but I timed with mmap and with fopen over several megabytes of input, and there's no difference (if anything, mmap can be slower if the window size is too small).

I don't actually allocate lots of small objects, which would make sense to use my own mem allocator. I've done that in another project, but here I only do maybe 5 or six mallocs in total, although there can be two or three reallocs max over the lifetime (doubling each time).

It's not like the program is slow, it's just that it seems weird to have so much time spent in system calls, and I don't know how to track it down.

Profiling (with gprof) isn't too helpful. It flags some functions I spend a lot of time in, but I suspect it's I/O(*), and that only accounts for at most a third of the total user running time. I can't really profile thousands of instances each running for a fraction of a second :(

(*) If I run the same code on one very long piece of input data, then system time practically vanishes. I really think I lose most of my time in process startup and shutdown with this.
--
$E(X_t|F_s) = X_s,\quad t > s$

[ Parent ]
Hm. by i (3.00 / 0) #26 Wed May 05, 2004 at 02:43:28 AM EST
Try prelinking, then. Or profile with a real profiler. Maybe oprofile. I'm currently fiddling with it myself.)

[ Parent ]
aha! by martingale (3.00 / 0) #27 Wed May 05, 2004 at 02:51:43 AM EST
Does prelinking work in C? I thought it was a C++ thing?

But I should definitely try statically linking the utility. That's a very good idea.

I'll keep an eye on your diary if you find other profilers. I'll try that one probably tomorrow.
--
$E(X_t|F_s) = X_s,\quad t > s$

[ Parent ]
update: static linking has no effect by martingale (3.00 / 0) #31 Wed May 05, 2004 at 04:53:08 PM EST
I tried linking statically, but the gain is about 10 sec over 6 minutes. Not worth bothering with. Oh well, next I shall try OProfile.
--
$E(X_t|F_s) = X_s,\quad t > s$
[ Parent ]
I have no idea what you can use. by hulver (3.00 / 0) #23 Wed May 05, 2004 at 02:00:05 AM EST
Wouldn't it make sense to change the utility so that it can be run just once, rather than tens of thousands of times?

That way you're only allocating the memory once.

Of course, knowing nothing about what it does, that may not be possible.
--
Cheese is not a hat. - clock

[ Parent ]
it's for testing by martingale (3.00 / 0) #24 Wed May 05, 2004 at 02:17:50 AM EST
The utility makes a decision, and I need to test the proportion of errors over tens of thousands of decisions. I could simply add a loop and run everything in a single instance, but that's not how it would be used in practice (and moreover, running it thousands of times is also a stress test. Only takes about 15 minutes altogether.
--
$E(X_t|F_s) = X_s,\quad t > s$
[ Parent ]
try it by jacob (3.00 / 0) #28 Wed May 05, 2004 at 03:23:11 AM EST
but I'd put my money on the lazy strategy being faster. The kernel always releases all the memory for a given process the exact same way (for each page, zero it and put it on the free pages list) regardless of whether your malloc/free implementation thought it was in use or not, so freeing the memory explicitly will just be extra work that takes time but doesn't have any extra effect. If there are a lot of separate things to be freed, that work could be really significant.

That's my guess, anyway.

--

[ Parent ]
Hrrmmm... by ti dave (3.00 / 0) #29 Wed May 05, 2004 at 03:23:48 AM EST
Can't wait to read fraise's next diary.

I don't care if people hate my guts; I assume most of them do.
The important question is whether they are in a position to do anything about it. --W.S. Burroughs

Presumably, by ambrosen (3.00 / 0) #30 Wed May 05, 2004 at 03:35:03 AM EST
she'll not feel much like writing one. Or maybe she will. Who know?

[ Parent ]