The Citrix set-up was failing but I got the other three virtual servers built and running, and I installed various tools like Ethereal and Sysinternals Monitors.
We figured out Citrix was failing because the dum-dum-doody-head version 4 software is language-dependent and the German Windows 2000 server we had wasn't really German but rather MUI; installed in ENU (US-English) it was then "switched" to German. There are huge differences between base language and MUI installs. I didn't get out of the office until 9pm.
I arrived at 7:15am and spent half the day just trying to find the damned German Win2K image. MSDN no longer has Windows 2000 on-line so it was up to the lab guy in England to search for old MSDN CDs. Once they were found, loading started but configuration took ages because he doesn't speak German.
Then there were problems getting the server into the Active Directory. By 3:30 we we'd managed to complete the OS install. Wolfie, the customer's tech guy, started installing Citrix through a WebEx session running on another virtual server which I was controlling through a Terminal Services session. This was more problematic because the physical server was on a different rack and mis-named: we'd had to change the host name to the original machine's name due to Citrix binding the host name and OS language into the license key.
At 7pm I walked to Lidl to get a bottle of vodka -- it was going to be a long night. I was drinking 60-40 vodka&Cokes as I worked with a contact on other parts of the test build.
At midnight the name was sorted and the software copied over but we couldn't install it until someone in IT could bring the machine back into our domain. I went home. The vodka bottle was empty but I didn't notice any effects until I tried to do some sudoku puzzles on the train. You know things are bad when you manage to put three sevens in the same box, but on the plus side, that was me coming out of work mode and finally relaxing. I hadn't been making such mistakes an hour earlier and Wolfie's praise in some later mail confirms this.
Back in the office before 7:00a.m. I was able to start testing. There were more conf calls. I had to again remind the guy from Citrix that the fault was on them, and not us or Microsoft. It doesn't matter that you can reproduce the error over RTP without Citrix 4; $OurBigApp worked in Citrix 3. It didn't change, the Windows environment didn't change, only Citrix changed.
At noon-thirty or thereabouts we were up and running. There were more WebEx sessions to terminal servers and Citrix was installed around 3:30.
At 4pm, Wolfie prepared for the final round of testing as I wrote out the protocol for the test based on a recorded session of all the crap I'd done on the Citrix v3 system.
And then it happened. $OurBigApp wouldn't start. It would crap out just before the log-in finished, ad it did so with error messages no one had ever seen before. Wolfie had to catch a plane at 5pm and called it a day. I contacted our California people for help, went home and worked from there with a colleague until midnight. Nada.
I woke up early and went right back to work on my laptop. I tried running the application locally on the server and got the same error so I knew I had a problem with $OurBigApp on the application server even though nothing had been changed in the past two days.
I checked out another machine and loaded it again with the same environment. That took hours . Since it was the weekend no lab admin was there to bring the machine into the domain so that SQL Server could start, so I figured I could run the server on this machine and connect to the database on the fuX0r3d machine. That worked once I found all the connection settings.
At 5:00pm it was time to go to the bar and work my shift. I was in no condition to work on it at 2am when I returned, but when I woke up four hours later, I was logged back in trying desperately to finish building the environment and have the tests done by Monday morning.
I'd logged into $OurBigApp from the wrong box and the application worked with my new set-up. That wrong machine was Win2K. I realised I needed to put the XP SP2 patch on $OurBigApp. It took a few hours to get the files over from the US and onto the box. I gave up Sunday afternoon and took a well-deserved few hours' break.
I woke up at 5:30 and was in the office before 7:00. I finished installing the patches, confirmed it had completed correctly and things were starting to look better. There was a small problem because I'd missed a data connector but then -- BAM! -- it was connecting.
I fired up the client again and -- BAM! -- it failed. I was struck by another revelation: Windows XP updates. I spent the next hour downloading the damned "Jen-you-whine Ass-vantage" crap and the fixes afterwards, all the while fighting the installer to prevent it from including the 912812 rollup. Do what I want, not what you think I probably want!
At 8:30am I was done. The only problem was that I couldn't connect to the Citrix Server through a terminal session, but I was able to run $OurBigApp normally from the XP box. So I tried a session through Citrix.
Big surprise: it failed. Some sort of Java error. I sent a message to Wolfie hoping to sort this before the 9:00am conf call. He sent me some info and screenshots back. It can only be done by connecting to the Citrix Server... which we couldn't do because Citrix requires that the server be in Applications mode.
We managed to set up a Citrix session on the test client so that we could get back to the Citrix server and everything was available. We performed the normal test and it was normal. Then we tried to do the test over Citrix and started getting weird failure messages. One of the messages was really odd and we tried Help: About. IE 5 SP1. D'oh! I left for my bar shift as Wolfie started installing IE6.
I arrived this morning to half a dozen E-Mails from different people all with more questions just about this particular Citrix problem. They'd searched through our tickets and noticed my name associated with most Citrix issues. When I switched windows to the virtual server with the client, I saw Wolfie'd reproduced the problem. It had only taken us a week!
The 10am conference call was long and tedious. We have a good idea what's behind the problem and the guy from Citrix admitted to certain changes between versions 3 and 4 which could cause this. It's about damned time. He only did so after we confirmed the primary method we use for popping and hiding windows.
All I have to do now is perform the tests following the same protocol to compare what's being sent or not sent. And then I get to do it again with both Ethereal and some DLL explorer/watcher tools. That and answer eight other tickets which need a response by today.
I don't even need to do that! The customer and Citrix are writing little scripts to pinpoint the hidden window call method usage and why Citrix
sucks screws up. Looks like this is just about done and I'll have one big write-up which I can cut and paste to eight different tickets. If this happens by the end of next week I'll be doing the w00t dance.
Tomorrow I'll be around 30,000ft over the Atlantic after a brief stop in Paris. Considering the number of hours I've put in since last Wednesday I made it clear they would not charge me a day off for the flight. I haven't even done the laundry much less started packing. I'll spend a lot of the plane ride slogging through Ethereal logs, with a couple breaks for The Queen's Jewels.
I hate Citrix thiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiis much. Times a-hundred. Plus infinity.
|< Oh, mother. | BBC White season: 'Rivers of Blood' >|