40 posts tagged “nerd fu”

I remember when I first saw the EDS Catherder's commercial. It was 1999 and I was in my little apartment in Oceanside, with 3 cats and a large bottle of white zinfandel and a coffee table that I was supposedly putting together. I paused, with a screwdriver in one hand and a glass of the awful pink stuff in the other, and watched the whole thing, and thought to myself: there is no way managing people or projects is really that bad. Then I promptly stripped the screw in the corner of the table; it still is a bit wobbly on that end ten years later.

Fast forward 10 years and Catherding defines what I do at work, each year, from Thanksgiving to Martin Luther King Day. These holidays have nothing to do with the actual projects, that just happens to be when they fall.

I've blogged about this before:every year we get together and redefine our geography so we can redefine our productivity. This year I thought to myself that if I could just take on more of the process, I could make it work better! I could! So instead of doing just the fiscal bit, I took on the geographies as well. I made use of technology (MOSS 07, MapPoint 2010, SS2k5 and 2k8) and screenshots and time, and created a simple, logical, path for people to follow (13 people, to be precise) with plenty of lead time for them (3 weeks).

The first two weeks there were encouraging thank-yous and complements on what I had done by all. By week three (that would be the last week) there was rising panic, which I quelled by individual half-hour mentoring sessions. By the end of week three I was having to go back and re-do parts of my work because others had not decided to look at things until the very end. In one instance, I had to redo a couple of worksheets (at about an hour apiece) THREE TIMES for the SAME PERSON because he couldn't quite figure out what he really wanted.

Each year I come away with this feeling spent and wrung and awful. Each year I vow next year will not make me do this, and that I will make things smoother. What I have discovered is I'm making the process more accurate and more simple: for everyone else. In the meantime, my sanity is slowly flushing down the toilet as I process the monetary drives that run into ten (count 'em) digits. Before the decimal point.

I could do with a shiny fraction of that, methinks.

The C had a rough day at school the other day and, when we were reviewing his day, started hitting himself. That drives me nuts, and I usually beg him to stop or admonish him. It's his way of punishing himself, and it is disturbing to watch and more disturbing to know your kid is hurting himself.

So I figured I'd try my parent's tactic: to talk to him at my level and let him know how I deal with pooey days. Because guess what? Mommy has rough days too. We talked about how if I'm overwhelmed I go into Tolga the Data God's office and whine and plead (true), and the C points out that I'm saying that just to make him feel better. I'll take him in soon to clear that up.

Then I talked about what I do to make myself feel better after a rough day: I... clean.

It was like someone opened a big, gaping window into my subconscious and out came all the smoke and other awful smells: holy crap, I'm an OCD cleanfreak because I'm unhappy with myself. Well, no, not quite that drastic. But I clean because it is something I can control: project isn't going to ship because skiplevel changes everything, ON SHIP DAY? Time to wax the floors; at least that will be awesome.

I'm coming to find out that as a parent you find out at least as much about yourself as you do about this junior human you're in charge of. I've also discovered I'm really, truly, not that good with change.

Unless it's change I know is coming.

You see, if I know a project is going to be riddled with change and uncertainty, I am Totally Copacetic. I will contingency plan and be smug when plan number 457 comes true (out of plans 1-13,492). This explains why the first few (15 or so) X changed tactics or things got shirty with him I freaked out: this was supposed to be a stable, non-changing thing (once divorced). Now that I know it won't be, the latest cryptic email or snyde comment bothers me only slightly less than the newest chin hair to be tweezed.

Did I mention my skiplevel changed the project on ship day? That was this Monday.

It shipped with all the changes Wednesday. And my boss thinks I'm awesome.

Disclaimer: I do not have Windows 7. The statistical likelihood that I will have it in the next year or so is equivalent to me being descended from Anastasia. You know, the one who died in vain according to Mick Jagger. I'm moneygrubbing and my laptop is 4 years old and I won't replace it for another year or two.

That said, I have long been a fan of the Get A Mac ads, because they were funny and punchy. They would playfully exhibit the differences (both in purpose and daily use) of each OS and it was at a level that was "all in good fun". The latest "Get A Mac" ad, though, is weak.

And here's why:

Every single previous ad brought out a specific feature, issue, or benefit that Mac had over Windows. Maybe it was removing the myth that only Windows OS could do business. Maybe it was exploring Mac's graphics/arts-friendly side. Maybe it was antivirus issues. But each and every ad had a point.

Until now.

This ad ("Broken Promises") is so generic it threatens the whole genre; there isn't a specific target except that each Windows release has promised to improve on its predecessor. Well, I would think that Mac's OS' have offered the same (after all, if there isn't improvement, then what is the point?). It's a below-the-belt punch, and it shows that Apple is truly threatened, or at least perceptually threatened, by Windows 7.

This bothers me on a couple of levels: one, I do have some significant friends and family who busted their @ss on Windows 7, and rightfully so: I hear it's a sweet ride. Two, I'm all for competition and playful (or barbed) banter: but it needs to have substance. The latest Get A Mac ad is the equivalent to Churchill retorting "I know you are but what am I" to any of his foils.

The irony being that it sends me down the path of looking to a Win-specific platform next year, and not switching to Mac.

I'm a simple pleasures kind of person. A decent glass (or two) of red, a warm beach, a pleasant vista, a home cooked meal. Each of these tickle my fancy. (My fancy isn't all that fancy, and I am distinctly "utilitarian" -- according to GH -- and I *like* it that way.)

Also on the list: solving a problem. Specifically, data problems.

I work with data for a living. I have worked with data for a living now for 5 years at BTCo (and 2 years beforehand in my old job), and in all my time I have never had angst as awful as the latest Feed. You see, BTCo owns some smaller STCos. I work for the strategic department within BTCo that has to define 1. how well ALL of BTCo is doing, and 2. how to make it do more well. Ergo, I play with data from all of BTCo, including the STCo's. It doesn't matter that we just acquired the last one a YEAR ago, I need to know it and own it.

And I totally would've, hadn't it been for the last SEVEN MONTHS troubleshooting the feed from this latest. We'll call them VD. I took on the VD feed when my boss (who is, and I mean this sincerely and with zero amount of snark, a Data God) threw up his hands in disgust. Yessiree, I was gonna solve this problem and make it all good. Thus ensued weeks upon weeks (and eventually, the aforementioned 7 months) of 6am phonecalls with Rome, assertations and pleadings regarding basic data hygeine, earnest reviews of input and output, and a lot of hair pulling.

I have learned that if the nice man tells you it's a duck, asserts it's a duck, and swears upon the life of his grandchildren it is a duck, and it does not quack/is not yellow/does not waddle, it's not a duck.

I could detail all of the problems with the data that I experienced, but it would mean nothing. All of it could've been avoided if they sent me their code (as requested) as I had sent them mine. All of it could've been avoided if they actually looked at the spec (and sample, and sample situations) I provided. All of it could've been avoided if they were willing to concede, at any point, that perhaps their data feed was flawed.

Tonight, after seven long months, I have solved the problem. I have figured it out. And yes, I'm patting myself on the back so hard that I've actually managed to throw it out. (Actually, I did that last night, but that's another story. Getting old sucks.) 

The Data Dominatrix has bitchslapped it into place, and it ainta-neva-gonna fugghedaboutit.

I sit in the corner, pleasantly plugged in and eyeballing the sweets case. I am at Herkimer Coffee, on Greenwood. I am here because the C has Theater Camp about 10 blocks from here and it makes no sense to drive the 20 miles back to Bellevue, only to turn around in 3 hours.

The early morning crowd was the standard pre-work coffee crowd plus a little bit of urban mommie light. They've long gone and now it is but three of us, either ignoring or enjoying the loud music and tending to our business.

You'll have to pardon me, I'm terribly smug; I've solved my data issue *and* devised a nifty (and almost incredibly efficient) way to test it.

Or maybe that's the coffee talking.

Today I participated in a Facebook multi-post where I had to type "[DD] likes to" into Google search and come up with the first 10 answers. One of them was, and I quote: "[DD] likes to tease Evan that he is merely unimaginative, stuffy before the possibilities of life, as obdurate as a cranky wombat."

Whomever wrote this, I owe you a drink.

First, I have read the word obdurate probably once in my life and ignored it as it was most likely used in such a context that I could get the general gist of what the author intended, without being so bothered by, say, having to actually look it up. Coming across this required I learned myself a new word: Obdurate.

From Merriam-Webster (m-w.com):

About two days ago I got a frenetic email from someone about the figures for the Balkans doubling in January.

Welcome to my typical day.

The end user is stationed in Germany, which means as I arrive to work she is leaving; there is an awkward lag then between question and response. The crux of the issue is this: If she runs a report from January 07-January 09, the January 07 figures are doubled. This is not to be supported!

This sort of thing comes to me, along with the "run this data really quick for the CEO and by the way it will be published in the New York Times so do make sure it's correct". You know, no pressure.

At any rate. The report they are referring to has two methods of execution: one against a series of tables that have "this year" and "last year" data side by side (so you can do year on year comparison in an easy fashion), and the other executes against a series of tables that do not (because these tables use up their extra space for massive more amounts of detail).

After about 5 hours of troubleshooting yesterday, I managed to discover the following:

• It only happens when we execute the more detailed tables.
• It only happens when we execute a date set longer than 12 months.
• It is happening within the first two aggregations in the code.

The code itself is about a thousand lines long -- there are if/elseif statements to do things like check to see what type of data the user ordered, what sort of timing they ordered it by, how detailed it needs to be -- so my "problem area" is somewhere between lines 50 and 110. I stared at them uncomprehendingly for about an hour yesterday.

In the middle of the night I had an epiphany of how I would test, but I didn't reach for my notepad and so now I'm back to staring at code.

[UPDATE]: Here was the solution to my problem. This will be interesting to you if you are interested in SQL, and not at all if you are not, so skip this bit if you are not:

Let's say you have a data point that you want to look at in a year-over-year perspective. Call it, say, Money. Money is a metric and you can sum() and count() and avg() money all you want. Money has dimensions, let us (for this example) give Money two dimensions: Date (datetime) and Source (varchar -- aka text). Let's say I want to look at the Money, by month, from January 2008 through January 2009 inclusive, and I want to look at the year-over-year effect.

The calculation we will use for Year-over-year is (This Year / Last Year)-1, expressed as a percentage. So if This Year  Money is 300, and Last Year Money is 150, then the percentage for year over year is (300/150)-1= 2-1 = 1= 100% growth.

In order to get our base data for the months, though, what I like to do is run all of the months into a table, and then link it on itself, like this:

select sum(Money) as TotalMoney, Date, Source

into #temp

from MyTable

where Date between '1/1/07' and '1/31/09'

group by Date, Source

select coalesce(cy.Date,dateadd(mm,12,ly.date)) as Date,

coalesce(cy.source,ly.source) as Source,

cy.TotalMoney as ThisYearMoney,

ly.TotalMoney as LastYearMoney

from #temp cy

full outer join #temp ly

on cy.date=dateadd(mm,12,ly.date)

and cy.source=ly.source

where coalesce(cy.Date,dateadd(mm,12,ly.date)) between '1/1/08' and '1/31/09'

And that would work just fine. But when I sent my code to our Data Gurus, they broke it into two inserts: one for January 07-January 08 (the "last year" of my data) and January 08-January 09 (the "this year" of my data), ostensibly in order to make it faster.

If you can see the problem with this, you should consider a career in SQL.

It took me a bit, but January 08 is doubling because it is getting pulled in with each process. I TOLD THEM they should've pulled it in one process, but do they listen to me? No. 

----

On the flip side, today is the 14th day of the 2nd month. 14/2=7. I walked 7 blocks to Top Pot (from my parking spot), to stand behind 7 people. 7+7=14 or 7(2)=14, either way. I think I am going to write a daily post about the numeric coincidences I run across (if I can), because I seem to run across them a lot. I'm sure they're meaningless, but they are fun :)

One of the things companies like to do in recessions is look to their (remaining) employees for ideas on how to save money and/or spend a little money in order to make a lot of money. Behold, my team!

My team is responsible for strategic data provision for analytics. Take out all of the buzzwords and the literal translation is that we are the number runner geeks for the empire. We control the data, and it's fun. It's also very intimidating to a non-technical person.

About 70% of our audience is non-technical. They're salespeople, designed to get our company out there and increase the number of contracts we have and in general make us the best in the business. They were not hired for their Excel pivot table creation skills, they were hired for their smile and handshake abilities. Therefore most of my team's brainstorming is how to get data from A to B and make it easier for the B.

We've come up with a really sweet concept -- using Share Point, an MS product with possibilities, and interfacing/bastardizing it with SalesForce, Reporting Services, and other fun things -- but naturally it is dependent on our company's ability to expand our Share Point environment. Today I am playing the devils-advocate game of, "What happens if we don't get what we want, and we barely get what we need?"  I suspect we can still deliver, in phases, using a dotNet platform and doing some truly creative coding, but it is an ambitious goal for a development team of 5 people who, most of the time, are running adhoc queries and creating reports to keep the emprie alive. It's particularly ambitious since the announcement regarding no merit increases; we are asking people to work harder and in new technologies for the benefit of the greater good and at the same rate. In a very odd way, I think this works with my team: the chance to branch out from their core skill set and get what will be a pretty slick app under their belts seems to have them excited, and it's infectious.

Or maybe that was the cupcakes I brought in this morning.

On Saturday, at brunch, one of GH's friends (well, I suppose she's one of mine now too so I'll have to give her a name -- let's call her Melissa) debuted her "list". The idea of a "list", you will recall, is to identify your deal-breakers and nice-to-haves in a partner, and then that helps you weed through the masses of potential suitors. I have my own list (geek hint: use "list" as my tags to see!) and it worked fairly well, GH meets at least 90% of the requirements (and all of the mandatory ones). It's not so much the idea that "if you build it they will come" but more "if you build it you will easily identify choads and give them the bird rather than the bed".

So the Ladies Who Brunch assessed Melissa's list and identified one or two addendums but nothing to remove, Melissa's got her shizzle together and doesn't have unrealistic expectations. I fully expect her to go into dating and have her swath of crappy dates, because it's a bonding thing with her girlfriends and she needs to have them, but once she's over that she'll find Mr. Right, or at least Mr. Pretty Darned Close and that, my friends, is the bird in hand over the two in the bush. No pun intended.

Filtration is good.

Think about all the things you filter: news (you either go Fox News or CNN, usually), data, information, your water, incoming work, outgoing work, the things you say, the things you do, etc. Everything goes through some sort of sieve-cum-processing system wherein you don't simply regurgitate what you've had in: GIGO may apply but not really with humans.

Filtration happens a lot in data.

There are, in the geek world, a lot of ways you can filter your data: going only to certain tables, inner joining (rather than left or right or outer joining), and then the ever-popular 'where' statement. "Where" is where it's at: the one line in db development where you get to say "I want chocolate and fudge and not almonds", because almonds kill me. 

(Yes, I'm getting my epi pen. I digress... still!)

There are many, many, many, many, many many ways you can filter stuff using that 'where' statement.

Before I start a treatise on this, please understand it's about to be all about DB's and no longer about relationships, and is useful if you like.. um... SQL.

Ok -- So you have your data set and you want to filter it by churches or very small rocks. First, find the field you want to filter. Let's use that initial example: "Things that Float".

Your select statement may look like this:

Select ThingsName

from MyDatabase.dbo.TableOfThings T**

Which gets you a  list of all the things in your TableOfThings. Now, a perfectly normalized database would have a boolean -- a field whose sole purpose is to have a 1 or a 0 -- that lists whether or not they "float". It may be called the "FloatBool" or somesuch, 1 means YesItFloats and 0 means NoItDoesn'tFloat. But let's say your db was built by the same sort of mentality that suggested "churches" float in water, and you'd have just the name to go on.

When watching Python, you see that things that float are suggested as "churches", "very small rocks", "cider", "gravy", "lead", "witches","cherries", "wood", and, finally, a "duck". Being rational, reasonable humans as we(?) are, we can say that we know for sure that witches, wood, and a duck float. I have doubts about the rest, please submit your scientific process and I will critique in light of the results.

So, if your TableOfThings includes Churches, VerySmallRocks,Cider,Gravy,Lead,Witches,Cherries,Wood, and a Duck, you could disseminate things that float (without that handy FloatBool) thusly:

select

ThingsName

from

MyDatabase.dbo.TableOfThings T

where

T.ThingsName like '%Witches%'

or

T.ThingsName='Wood'

or

T.ThingsName>='Duck%'

or

T.ThingsName in ('Cider','Gravy','VerySmallRocks') 

Wait, wait, what's with the %?? I don't want a percent of a witch, I want a real, honesstoGod, wart on the nose Witch!

Aha! But what if the person who put the "Things" in put in Duck: Mallard? Or Witches That Kick Ass? You would need your filter to be smart! Yesiree, Bob, a smart filter can make your life easy.

Using the "Like" statement says, "find anything like this. Like meaning EXACTLY LIKE this, unless there's a %, which means do everything exactly like the actual words and then use this % as a wild card for one or more letters".

So the statement, Like '%Wart%' can catch 'Wart' and 'Warts' and 'Warts Are Sucky' and 'Warts mean you need to go swing a dead cat in a cemetary, you know'; or it can mean 'I love Warts' and 'I love warts that are on hairy palms'. Because the % is on the fore and aft, as it were, it only cares that *somewhere* in thingname is the string of characters that spells 'wart'.

Ok, what about that>=? What the ?#@?$ is >=?

That, my friends, is an easier way to do the "like" thing, *if* you don't need it to be flexible on the first part.

For example, thingsname>='Deb%' will get you anything that is equivalent to 'Deb' (uh...'Deb' is pretty much it) and anything that starts with 'Deb' that is alphabetically past that. So it won't get you anything that starts with 'Dea' but it will get you things like 'Debutante' and 'Debater' and 'Deb is the meanest beeyotch ever'. 

Lastly, the "In" statement: it's good if you know what you're dealing with (e.g., I know the db dev and he would never put in I love Warts but only Warts) and you want to give it a list to look "in".  

Ok, you say, so if I have a table of say, sales in dollars by person name, and I want to filter by anyone whose sold more than 1000 bucks, I'd just do >='1000 bucks', right?

Whoa, nelly. Not so fast.

Those little '%%' things, and 'like', work great with text. Not so much with the numbers. If your field is based on an int, decimal, float, money, double, etc. (any numeric type value) you will want to use numeric qualifiers. In that case, you are limited to:

=. e.g., where fieldname=200.

>. 'greater than', e.g. where fieldname>200. Yields 201 and up, but not 200.

>=. 'greater than or equal to'. e.g., where fieldname>= 200. Yields 200 and up.

and then of course < for less than and <= for less than and equal to. "In" also works here: in (200,300,400). But again, you have to know what you're looking for exactly.

But perhaps the really coolest thing about filtration is you can compound it. No, you are not limited to picking just one thing and having done with it! Like Melissa's list, you can pick a bunch of things: and the more you do, the narrower your results set is likely to be.

So for example,

where

chestmuscles='defined'

and

iq>=130

and

dentalhygeine='good'

and

hasoverbearingmotherbool=0

will yield you less results than, say,

where chestmuscles='defined'

and

iq>100

and

dentalygeine>='g%' (cuz this will get you grand, great, good, but may also get you grotesque)

and

hasoverbearingmotherbool>=0

Just sayin'. Keep your where statements loose.

** What is that weird T doing there? It's called an alias. When later you have to refer to that table, you don't want to carve AAAAARRHG -- I mean, "mydatabase.dbo.tableofthings". It's unweildly, and it gives headaches when you've had too much caffeine and want to refer to things easily. By putting the T there (or an M, or a X, or Tot, or Tomt, or MyFancyAlias, which really defeats the purpose but illustrates the intent) you can shorthand it.

The C's first day of Kindergarten was apparently a success. He enjoyed himself, the after-school care counselor said he was a "wonderful boy", he did his homework, and was eager to return. Yay for good beginnings!

I had my review yesterday and my boss still likes me. Yay for good continuances!

I went for Run Four yesterday, which I ran in its entirety, only 2.2 miles. However, those 2.2 miles were uphill and downhill, with a total elevation gain/loss of 200 feet. I live on a freakin' big hill, and boy my ass is feeling it today. Later this afternoon I'll run up the hill and around the foo-foo neighborhood to my north, here's hoping my new earbuds have arrived.  Yay for tunes while pounding pavement!

Last night GH and I watched Manhunter. I saw Red Dragon in the theatre and thought it was quite good; I like Ralph Feinnes and Edward Norton and of course Anthony Hopkins. However, now that I've seen Manhunter, I see what the fuss was about at the time (many people skwawked about how it was a "terrible remake"). I think I have to agree with them, my rationale being:

• Brian Cox is a lot squickier of a Hannibal Lecter. I had never seen him when he was younger, and that slicked hair and catepillar eyebrows combination already started the skin a-crawlin'.
• While I think Edward Norton is a brilliant actor and has a good asshole range, he didn't seem as mentally or emotionally tortured as William Peterson, his counterpart in Manhunter. That, and I don't think Edward Norton could pull of pink short-shorts as well.
• Tom Noonan, who played the Tooth Fairy, was much creepier than Feinnes was.  I mean, the forehead alone...
• Two Words: Dennis Farina. Yay for Italians!

And that was how *my* Wednesday went.