Archive for the ‘Programming’ Category

Multiline Search and Replace in Visual Studio

Tuesday, February 26th, 2008

Using this is a little weird. It’s basically a plain text to regex translator. A box will open and you paste your lines of code. Then click “Find”. It will convert the text into one long regular expression and then you click “Find” again. It has Find, Find in Files, and Replace. Could be very useful for refactoring.

I was cleaning my older VB .NET project some time ago. I just wanted to “refactor” some code to look more readable. All I needed was several replaces of multiline text. VS 2002, 2003 nor 2005 is unable to perform multiline search/replace. So I wrote a macro that can do it.

Multiline Search and Replace in Visual Studio

H: Multiline Search and Replace in Visual Studio

Friday, February 22nd, 2008

Using this is a little weird. It’s basically a plain text to regex translator. A box will open and you paste your lines of code. Then click “Find”. It will convert the text into one long regular expression and then you click “Find” again. It has Find, Find in Files, and Replace. Could be very useful for refactoring.

I was cleaning my older VB .NET project some time ago. I just wanted to “refactor” some code to look more readable. All I needed was several replaces of multiline text. VS 2002, 2003 nor 2005 is unable to perform multiline search/replace. So I wrote a macro that can do it.

Multiline Search and Replace in Visual Studio

IE Developer Toolbar

Wednesday, February 20th, 2008

This has been really helpful. You can see the web page and the HTML that made it at the same time. There is a mode that highlights the HTML for any element that you click on.

Another very useful feature lets you select some controls on the page and do a View Source on just that part. This works even if various page elements got there from Include files. Normally, you can only do a View Source on the top page.

Internet Explorer Developer Toolbar
from IEBlog

How to Calculate the Character Width Accross Fonts and Points

Tuesday, February 19th, 2008

Ok, this is going to be long and technical. I’m writing it up for all the fools who, like me, have to figure how many pixels wide an Input field on a web page should be when the font and size are variable. Instead of letting the browser figure out the size.

This was a hideous problem to resolve, because it seems seductively simple at first. Just multiply a constant by the number of characters in a field. Give that a shot. The code I was replacing use 8 px/char on a Verdana 8pt. It doesn’t work. Short fields are too narrow and long fields are massively wider than necessary. And this strategy doesn’t account for other fonts the user can setup like Courier New for pt size from 6-18.

Each font has a unique character width. Each pt size has a unique character width.

My solution has a few parameters. The exact string is unknown. Font can be anything that the user can type and the system recognizes. Pt size again can be anything. Width is in pixels set on the Style attribute of an HTML Input tag rendered in Internet Explorer.

The system has a few defaults and I have a few favorites. So, I selected Arial, Consolas, Courier New, Georgia, Lucida Console, MS Sans Serif, Tahoma, Times New Roman, Trebuchet, and Verdana. On Point sizes ranging from 6-18, but preferably this is a function which can handle any Pt size.

Now, we see Windows, Word, and Internet Explorer solve this problem every day. The text box looks the “right” size. So, this problem has a good solution, because we see it and don’t even think about it. Perhaps the operation system will have an API or .Net call that will help us figure it our or do the calculation for us.

It does. And it’s called TEXTMETRIC In .Net 2.0 and earlier you have to reference GDI32.dll and call the API. .Net 3.0+ has a FormattedText class that seems to do exactly what we want. Unfortunately, this project is in .Net 2.0 and FormattedText is unavailable. There is yet one more way that will work in all the .Net versions. Make a Graphics object and use MeasureString to find the pixel width of a specific string.

So, lets take these one at a time. TextMetric can give you lots of information about a font. Of particular interest are MaxCharWidth and AveCharWidth. There are many ancillary measurements.

I ran a test program that would spit all of the Max and Ave(Average) character widths to a file. There is a large amount of variation here. For example, Arial 8pt has a Max of 29 and an Ave of 5. I thought I could make a functions that weighted the Max and Ave and generate a good result. This proved difficult and did not carry over to other fonts and other pt sizes. Each one would have to be guesstimated and the result might still need tweaking.

A lot of time was spent analyzing the differences between Pt sizes within one font. When you plot Max and Ave out, the graph is pretty close to a straight line. There’s some obvious rounding that causes a bit of stair stepping, but it’s not too bad.

What really sucked was the multiple characters. I had assumed that character width, on average, is constant. Yes, I know most of these are non-proportional fonts, but overall it should level out. Wrong!

This solution was not going to work. And analyzing the Max and Ave font sizes wasn’t revealing a formula that would handle any Pt size.

I did stumble upon an great Excel spreadsheet done by a college prof that will let you graphically perform a Least Squares Quadratic Analysis on a data set to determine a formula based on data only. This thing is awesome.

The code would be two layers of case statements. “What font, what Pt size, here is your function.” 10 Fonts X 12 Pt size = 120 possibilities plus a default. Ugly code. Ugly code isn’t like Ugly Betty or the Ugly Duckling. It never looks better.

The .Net 3.0+ solution with FormattedText looks great, but it can’t be considered and if this wasn’t fixed now it wasn’t going to be.

So, I rearranged my test program to use MeasureString. It spit out a comma separated file with all 10 fonts, 12 pt sizes, and string sizes from 1-79. The string was a ‘W’ at first, but became an ‘M’ after I read about em measurements. The graph I really needed was a 3d that showed the pt size and strings size, but I don’t know how to make one of these. And then give me a formula on the variables.

I do know how to calculate slope on a line and I can see that Pt size on the same number of characters is a line and not a parabola, sine wave, circle, etc. Dito with the number of characters on the same Pt size.

Slope is m = (y2-y1)/(x2-x1). (Didn’t think high school geometry was useful did you?). Slope represents the rate of change. If you know the Pt size and pixel width for 2 characters you can use slope to calculate the pixel width for 3 characters, just by knowing the Pt size. Since these are all lines the slope will be the same from 2 to 3 characters as 2 to 20 characters.

Lets look at Verdana 8pt. The Pt size is fixed and the number of characters are variable. The first pass of the formula works out thusly..

pixelWidth = fieldSize * mfor 8pt

Plug in the numbers from the spreadsheet generated by the test program to check the formula.

22.07639for 2 char = 2 * 9.26042

But, 2 * 9.26042 = 18.52084 and not 22.07639. Perhaps the formula needs some revision. Lets take the difference, 3.55555, and add it as a constant. The formula is now..

pixelWidth = fieldSize * mfor 8pt + 3.55555

Let’s jump up to a 10 character width and see if this still holds.

96.15971 = 10 * 9.26042 + 3.55555

96.15971 does not equal 96.15975, but it’s close enough. I think we found the formula for calculating the pixel width for Verdana 8pt.

Maybe, the m constant is not quite so constant. If we could get 1 formula for all Pt sizes instead of 18 of them that would really streamline the code.

Let’s get the formula for 7 pt. Here’s the data for 7 and 8 pt.

Verdana 7 pt

Num Char Width Slope (m)
1 11.21397  
2 19.31684 8.10287
10 84.13975 8.10287

Verdana 8 pt

Num Char Width Slope (m)
1 11.21397  
2 19.31684 9.26042
10 84.13975 9.26042

pixelWidth = fieldSize * mfor 7pt + 3.55555
19.31684 = 2 * 8.10287 + 3.55555

19.31684 does not equal 19.76129. The difference is .44445. A little checking against the other character sizes reveals the 3.55555 is variable on the Pt size of the font. For 7 pt it should be 3.11111. Actually, this constant calculates pretty easily to be pointSize * .44445 and this holds for all the fonts. I suspect that this constant varies on Character Set or Languages.

Lets plug this in and calculate on the 10 character data point.

84.13975 = 10 * 8.10287 + 3.11111

84.13975 is close enough to 84.13981. I think we have the new constants for Verdana 7 pt.

Is there a relationship between the change in the constants from 7 pt to 8 pt? We’ll start with a simple relationship. Calculate the slope of the slopes. This may not be true. The relationship could be a parabola or something else.

(9.26042 – 8.10287)/(8-7) = 1.15755/1 = 1.15755

You can quickly check this against the other point sizes for Verdana and see that 1.1575 is a good mfor all points. There’s a faster way to calculate mfor all points than comparing various point sizes on a font. I mean we have to do this 9 other times. There is a shortcut.

mfor all pt = mfor pointSize/pointSize

or to put it another way

mfor 8pt = 8.10287 + 1.15755 = 9.26042

for 8pt

1.15755 = 9.26042/8

So, it checks.

How does this relate to the existing pixelWidth formula?

pixelWidth = fieldSize * mfor 8pt + constant
where mfor pointSize = pointSize * 1.15755
and constant = pointSize * .44445

or the one formuala we have been looking for..

pixelWidth = (fieldSize * pointSize * 1.15755) + (pointSize * .44445)

The formula above holds for all the fonts. The constant, 1.15755, is unique for each font and actually varies by character. 1.15755 is based on ‘M’. ‘W’ will give a slightly higher constant.

The table below holds the constants for all 10 fonts. The formula is always..

pixelWidth = (fieldSize * pointSize * 1.15755) + (pointSize * .44445)

Font mfor pointSize
arial 1.14386667
consolas 0.7552125
courier new .82421875
georgia 1.2734375
lucida console 0.827475
microsoft sans serif 1.14388
tahoma 1.0579425
times new roman 1.2213575
trebuchet 0.9746
verdana 1.1575525

Here is the test program that generated the data in .Net 2.0 c#. Here is the best of the many, many datasets I generated. It’s a comma separated text file.

Sorry, I can’t list all the blind alleys I walked into. That’s a much longer and probably more useful article. There were 3 days of them.

Good luck with your project. Thanks to my friends for helping with a simple, elegant, one formula solution that fit all the parameters.

CH: The Years of Experience Myth

Friday, February 8th, 2008

This comes from trying to treat people like computers.

If (Resume A matches Job Requirements B) Accept(); Else Reject();


This toxic, counterproductive years of experience myth has permeated the software industry for as long as I can remember. Imagine how many brilliant software engineers companies are missing out on because they are completely obsessed with finding people who match– exactly and to the letter– some highly specific laundry list of skills.

Somehow, they’ve forgetten that what software developers do best is learn. Employers should be loooking for passionate, driven, flexible self-educators who have a proven ability to code in whatever language — and serving them up interesting projects they can engage with.

It’s been shown time and time again that there is no correlation between years of experience and skill in programming. After about six to twelve months working in any particular technology stack, you either get it or you don’t. No matter how many years of “experience” another programmer has under their belt, there’s about even odds that they have no idea what they’re doing. This is why working programmers quickly learn to view their peers with a degree of world-weary skepticism. Perhaps it’s the only rational response when the disconnect between experience and skill is so pervasive in the field of software engineering.

The Years of Experience Myth

CH: Every User Lies

Monday, February 4th, 2008

There are a bunch of ways to restate the paradox of the active user. Cooper calls it perpetual intermediacy. I think the easiest way to explain it is this: every user lies. Instead of asking users if they love your software– of course they love your software, it’d be rude to tell the people responsible just how mind-numbingly awful it really is– do what Gregory House does. Observe whether or not they use the software, and how they use it. Rely on that behavioral data to design your software, and not the lies of your users, however well intentioned they may be.
Every User Lies

Breakup a Super Long Line

Thursday, January 24th, 2008

These days lots of other languages are used to generate HTML; VB, c#, PHP, Perl, etc. They can make some pretty awful stuff. One example, is a ~1.5 meg single line of HTML. This is really unusual and I wanted to determine if it was messing up IE6/7. How do you break this line with a Find/Replace in Notepad++?

The textboxes in the Find/Replace dialog are single lines. Carriage returns are chopped off when you Paste.

Turns out it’s pretty easy if you turn on RegEx. Add your line of text in Find and add the similar line with a ‘\n’ at the end to the Replace box. Check ‘Regular Expressions’ and click Replace to watch it run once. Do “Replace All”. Voila, lots and lots of smaller lines.

One way to backdoor a feature..

Tuesday, January 8th, 2008

Wait for a while, till all the original developers are reassigned or resigned. Claim that something worked, be noisy about it, and have the political/financial clout. Of couse the feature you want never worked, but it’s been so long nobody knows the difference. Voila, a service pack will appear.

Wide vs Deep

Friday, December 21st, 2007

On not being a manager..

I’ve got this theory about what it’s like to be a manager and what it’s like to be a developer and which role suits a particular individual best, and I think it explains pretty well why I deeply, profoundly hate the former and dearly, truly love the latter.
Wide vs Deep

Where is my Googlish Database Searching for SQL Server?

Tuesday, December 18th, 2007

If you’re stuck using an internal application or maybe even an app that you company bought the Search capability probably sucks. I know every app I’ve worked has 1990’s search technology in it. We’ve all used Google and marveled over its ability to find what we wanted and Google isn’t even recent Search technology. It’s from ~2000 with some updates. Ask.com probably has the best and most recent search engine technology. It’s worth trying out if you’ve only every used Google.

Why do searches on so many applications suck? Most companies making products are cheap about it. They use the search technology that comes with the databases. What’s that? It’s either Full Text Search or Wildcards. Both suck and both are old. Wildcards are * and ? from the DOS command line. Is that old enough for you?

Full Text Search indexes the contents of certains fields. This probably could use a Google-like engine if Microsoft or Oracle chose it, but they don’t. And they should. What a great advantage one could have over the other, heh? Why doesn’t Microsoft package an old MSN engine in SQL Server? Say 3-5 years old.

If you want nice searching in your app, don’t complain to the developer. They can’t help you. Ask the two massive database companies why they aren’t innovating on this. Do they want Google to fund an Open Source initiative in which they donate the 2002 Google search engine? Does MS need another front on which they compete with Google? That’s what will happen if these guys don’t get off their duffs and innovate.

Scripting on Macs

Saturday, December 15th, 2007

Have you ever wanted the computer to execute a series of actions. Maybe all the files in My Documents should be syncronized to a backup folder after Word closes. Or you want to rename all the pictures your camera took. Or you want to rotate a whole bunch of photos. Or play a slideshow with one click. These task on Windows takes some programming skill. I’ve written programs and scripts to do three of these. Several hours of development and testing were involved. It was satisfying, but I was left wondering why such simple tasks are so difficult.

Particularly, the file renaming. I would think anyone with, by now, thousands of photos would want something more meaningful than DSC000023.jpg or 2007-05-12-0012.jpg, etc. The best you can accomplish with Windows is MyFileName (1), MyFileName (2),.. Since, we’re on this topic there’s one more thing that should be in modern operating systems. That is the ability to read the EXIF data from camera files. I don’t care if it requires a license fee. MS and Apple should pay it and move on.

While these activities require programming skill in Windows they do not on Mac. There are two ways you can script actions; Automator and AppleScript. If the program has the right hooks it can be manipulated by Automator. That gives this scripting ability to non-Apple programs.

The Unofficial Apple Weblog has started a series on Automation on the Mac. It should be a good overview of what you can do with Mac scripting and how to do it.

I am astounded that with all the copying MS has done of Apple, they have not copied Automator. Their tool that is roughly analogous is Visual Basic Script/CScript/Window Management Instrumentation (WMI), which has several long standing defects and which MS seems to have abandoned 5+ years ago. These tools are pretty much limited to the Windows Operating System and a few MS products.

Planning for Failure

Tuesday, November 27th, 2007

We recently had a defect that required a code change so big it was sure to cause lots of unpredictable baby defects. The thought of a never ending cycle of defects was nauseating. I knew I could not do this successfully. Or, to put it more accurately, “There was a high probability that portions of the system would be affected in unanticipated ways.” This change had to go into production between versions, actually, between maintenance releases.

How do you reduce the risk on something like this? First, draw a circle around the defect and fix the intended issue. Then draw a second bigger circle around the likely risks and add/modify features that mitigate them.

In this case encryption was added to lots of fields, which could confuse the old code that did not antcipate the encryption and might mistake the encrypted data for real data. So, the second additional feature was a kill switch on each field, a DontEncrypt flag.

This solution adds it’s own issues. The DontEncrypt code had defects to. In essence, two significant features were added at the same time. It gave me the willies at first, because I was afraid the DontEncrypt code wouldn’t work. Or worst yet, cause unrelated defects.

Now that we are in production, issues continue to arise from the Encryption code. However, we are able to quickly and easily fix clients by setting DontEncrypt flag to true. It’s saving significant development time and keeping the clients working more. We have yet had an issue, outside of development, with DontEncrypt or a defect that this flag doesn’t fix.

My point is that you should think ahead to that day when your feature is in place and users are reporting bugs. Ask yourself what you could have done to prevent this. Then remember that that day hasn’t happened yet and you can do something today to prevent issues.

Hacknot: If They Come, How Will They Build It?

Thursday, November 1st, 2007

Getting a job as software developer is just the beginning. It’s a stressful few months at the start. Not only do you have to deal with things like how you fit in the organization, who to take breaks with, who knows what part of the system, etc. You have to deal with the build process and getting your computer to a state where you can actually become productive. This chain of emails details an bad build process. I’ve seen pieces of this chain too many times.

If They Come, How Will They Build It?

To: Mike Cooper
From: Ed Johnson

Hi Mike,

I started on the AccountView project today. Can you tell me how to get the code and get started developing?

Thanx,
Ed
To: Ed Johnson
From: Mike Cooper

Hi Ed,

The code is all in CVS in the module called AccountView. Just check it out and you’ll be right to go. As you’ve probably noticed, we’re all using the Eclipse IDE here. That’s all you need to get stuck into it.

Mike
To: Mike Cooper
From: Ed Johnson

Mike,

Can you tell me the connection details for your CVS server? Will I automatically have access to it, or will I need someone to create an account for me?
Ed.

Programming Fonts

Friday, October 5th, 2007

It’s pretty much a given that programmers look at the computer screen all day at thousands of tiny lines of code looking for that one misplaced semicolon, parenthesis, or curly brace. How do we do it? We cheat.. a little anyways.

Most people don’t give fonts much notice. They can be enormously helpful in making the letters easier to read and making inconsistencies in blocks of text apparent. The first thing I do with an new development environment is copy my font, usually Andale Mono, and shrink the size one or two points.

One thing to point out for non-programmers. We use fixed width fonts on code. That way all the letters line up in a grid. Almost all the fonts in Word are variable width fonts. w and m are wider the 0 and l. The most common fixed width font is Courier New.

Jeff Atwood at Coding Horror has a recent article on programming fonts.

Can Your Team Pass The Elevator Test?

Monday, October 1st, 2007

Jeff Attwood has a great article that shows how much the programmer in the bowls of the organization knows about the product features they are creating. When you wonder, “Why would it work this way?”.

Can Your Team Pass The Elevator Test?