Deprecated: Function set_magic_quotes_runtime() is deprecated in /home/mwexler/public_html/tp/textpattern/lib/txplib_db.php on line 14
The Net Takeaway: Page 7


Danny Flamberg's Blog
Danny has been marketing for a while, and his articles and work reflect great understanding of data driven marketing.

Eric Peterson the Demystifier
Eric gets metrics, analytics, interactive, and the real world. His advice is worth taking...

Geeking with Greg
Greg Linden created Amazon's recommendation system, so imagine what can write about...

Ned Batchelder's Blog
Ned just finds and writes interesting things. I don't know how he does it.

R at LoyaltyMatrix
Jim Porzak tells of his real-life use of R for marketing analysis.







But its good for you... · 04/23/2008 05:57 PM, Personal Trivial

Europa Cafe, 1412 Broadway, NYC.

Small Salad plastic bowl, no toppings = $3.25
Ingredients (tuna fish, sundried tomatoes, chickpeas) = $4.50

Tax to total = $8.40

I guess that’s priceless.

But really feels more like a total ripoff.

Comments? [1]

* * *


IndexTools migrating towards Free... · 04/15/2008 05:34 PM, Tech

Dennis Mortensen, head of IndexTools, puts it a bit differently than I do… IndexTools (Yahoo!) Web Analytics goes FREE!

I would caveat this by pointing out that its not totally open to the world yet.

Further to this, it should be noted that Yahoo! (I should probably teach myself to say – we – at some point) does not intend to add any new partnerships or direct clients in the short to midterm, while we prepare for the next rollout wave.

So, current customers can stop paying, current agencies can stop charging for use (if they wish to pass through the cost savings), and we start the journey towards totally free. But if aren’t currently a user, you have to wait just a bit longer. Or cozy up to some of the great reseller/partner agencies, perhaps starting here.

But it’ll be worth the wait. Because the price is not the value of this offering: free is just the candy coating on top. Don’t judge it by free, judge it by what it can can do for you. Soon, you’ll wonder why you ever used anything else.


* * *


Can't Join Google App Engine · 04/11/2008 12:20 PM, Tech

I mean, yes, you or I will be able to join it in the future, but for the moment, I don’t know what I would do there. If you haven’t heard, look at Google App Engine. You see, even if you join, you can’t join from a database sense.

That is, it’s an impressive offering with all sorts of cool stuff, akin to Amazon’s services, and has glowing reviews (like this one from Niall Kennedy) but it’s missing one basic thing.

It has no relational database. That’s right, no joining of tables. Welcome back to the glory days of hand merging text files.

Yes, it does have a query language called GQL and there is a full API explained in the Datastore API section, but as they clearly say on the GQL reference page, “A GQL query cannot perform a SQL-like “join” query.”.

At the end of the day, they have chosen to use the classic Google datastore model that works for many… but that I still have trouble with.

Look, Map-Reduce is a classic approach (correct, Google did not invent it, but they sure made an amazingly powerful implementation of it) and there are a lot of things one can do with it. But I am not a good enough Object-oriented programmer to use flat datastores. I like relational databases. I guess I am an becoming an oldie.

It’s just disappointing. All sorts of things will port right over there (Python, Django) but to then have to work around this annoying deficiency seems like a complete missed opportunity. Imagine the boost Google could have given to, say, PostgreSQL, the last remaining fully independent open source database with a sizable user base. (BTW, if you like MySQL, you’ll LOVE Postgres).

It is just a beta/developer release. Perhaps they will roll out this stuff in the future; we’ll have to wait and see.

Or, that which does not kill us makes us stronger. Maybe it’s just time to learn hard-core objects, where the app describes relationships between entities and middleware manages the join process “behind the scenes” so to speak.

Either way, it’s still a nice offering. So, Amazon, Google… next would have to be MS, Adobe (Flash hosting, watch and see), and maybe even another large internet player?

PS: Lots and lots of articles on this, but the quickest example on how to work around this was Google App Engine: One-to-many JOIN.


* * *


Yahoo! Acquires Indextools · 04/09/2008 10:37 AM, Tech

(Note: I work at Yahoo!, and I’m jazzed about this acquisition. I am speaking here as a guy who loves web analytics, not as an official Y! spokesperson or anything. So, my opinion here may or may not be shared by folks at Y! and should not be taken as official company stance, policy or anything at all official, actually).

Lot’s of folks don’t know that Yahoo! has some of the premier web analytic talent in the world working here. From founders of companies like Accrue and Keylime through to folks working on analyzing usage data across the largest site in Internet history, we’ve focused on trying to turn data into insights, into something useful about people and their preferences and expectations, into knowledge.

But you can’t always do everything yourself. And sometimes, you come across some more people who “just get it”. So, today, I’m pleased to help announce that Yahoo! has acquired IndexTools. Please see Yahoo! Announces Agreement to Acquire IndexTools’ Analytics Business for the official announcement.

I first encountered these guys a few years ago when working with Tesco’s email at e-Dialog. Tesco, the largest retailer in England and one of the tops in the world, doesn’t trust their data to just anybody (trust me, it really took some time to get that data just for their email!). Even back then, IndexTools had some clever ideas around segmentation and report customization, but they hadn’t made as much penetration to the states. Nevertheless, they had set their sights on the “Omniture” market, and were growing quickly.

Now, they’ve expanded their client base around the world, along with agency reseller partners who use their tool as their primary analytic dashboard for their clients. The tool continues to be a very impressive mix: It has a strong segmentation component, including the ability to mix passed-in variables with web behaviors. It has easy drag and drop report customization, like an Excel Pivot Table online. It features well laid out reports and a pretty good sense of giving reports which are pretty helpful out of the box.

The obvious question you are going to ask: Does this compete with Google Analytics? Well, does BMW compete with Ford? Yes, they both sell cars, but they are attacking different markets with different approaches. There are some wonderful features in Google Analytics, and I use it here on this blog. There are things that Google Analytics does that Indextools does not, and vice versa.

But we came at this from a different direction. You see, Yahoo has a variety of ways to interact with partners and users: you can be a development partner building something around Maps or our other APIs, you can be an advertiser running rich media or search ads or microsites, you can be a small business running your store on Yahoo Shopping (and, yes, you can be a user using RSS or mail or other services: think about what analysis could do for them? Hmmm…) and across all this, we realized that we needed a capability which could analyze usage data across a variety of these modalities and help people working with our functionality to get optimal use out of it.

As part of being a partner, you have to give people ways to understand how their efforts with you are working. So, yeah, from a marketing analysis POV for advertisers with us (we do consider them partners), this tool makes sense; it delivers the power of Omniture for a fraction of the price. It links your marketing to behaviors on your site, with segmentation and other analytics approaches at your fingertips.

But it also will lend itself to the variety of other ways partners work with us: imagine giving that “maps functionality” developer an understanding of how their tool is being used. Imagine helping people using any of the APIs we offer understand how to optimize their site, drive traffic, and get better usage.

Google Analytics is a basic tool, and it delivers some pretty basic site analytics (though I like the testing component, it’s pretty advanced for a basic tool). It’s not aimed at the guys who really want to understand their data, nor is it aimed at the variety of ways people can work with Google (it ignores all that API stuff, for example). We see Indextools as the foundational start to understanding not just marketing and its impact on site behavior, but how to understand your online site usage to achieve your goals, be they commerce, fame, or just a mention on Techcrunch. Not many companies have the capability to deliver on that story.

In fact, we looked at most of the tools on the market today: all the majors, and many startups. Few had the combination of capabilities to deliver on this approach: strong event-driven data capability with an attractive reporting front end, and a back end which is flexible in its approach to meeting the analytic need of a very different web from the one Omniture, Webtrends, etc. started out measuring. Not to say that those tools suck (because they don’t, not completely), but that we were looking for a unique mix, and Indextools had it.

Go click around their site: Look at the functionality. Like I said, it’s certainly not Google Analytics; its more powerful and yes, a bit more complex on the interface (at this point). It probably won’t be free to everyone initially. In fact, it may never go free and unfettered; you may have to partner with Yahoo! to get access to this functionality (which might simply mean advertise with a search ad, but you get the idea). You’ll hear more about how things will fall out in the near future (BTW, the world changes on a daily basis, so keep watching the news (cough, hostile takeover attempt, cough)).

If you just want to get page counts, there are lots of free offerings, and Google Analytics is great at this. But if you want to see what it’s like to have access to Omniture-like capability without having to pay Omniture, you want to see how a data-driven approach to the world looks, if you want to see how analytics can improve every aspect of interaction on the web, then keep your eyes on what we’re doing with IndexTools.

I think you’ll start to see Yahoo! as more than a media company. You’ll start to see what we can do with data and technology (both invented here, acquired, and partners), things we’ve used in-house but are now starting to share with the world. You’ll see Yahoo! as a technology company. As an analytics company. And you’ll think about analytics in a new way because of what we’re doing.

Congratulations to Indextools!

PS: For some additional perspective, check out Eric Peterson’s post.

Bob Page (mentioned above) also posted about the acquisition.

Comments? [1]

* * *


The Ring · 04/07/2008 12:45 PM, Personal

I had a nice Saturday night. I got a skybox seat at Madison Square Garden to a sold out show. Food and drinks in the rafters, looking down through the hanging LED screens as strains of an instrumental version of Prodigy’s Smack My Bitch Up rose throughout the arena.

The circus sure has changed.

Yes, I was at the Ringling Bros. and Barnum & Bailey circus. Note that, officially, it is not a Circus, it is “a wacky and whimsical circus spectacular”. Anyway, we had the opportunity to join some friends in their box at MSG to see the show. You go in through a side entrance, up to a 3rd floor restaurant and bar, then a second elevator takes you to the 10th floor to go to the box.

The boxes are all about the size of an average NYC Hotel room. Each has a bathroom, 2 rows of seats (7 or so per row) with ledges for food/drinks, and a standing area. Ice, fridges, freezers, and a sink along one wall, a table for food on the other. The view is literally from the roof; you look straight out to the retired number hangers. There is glass in front of you, but it’s only half height, which scared us when Sam started running up and down stairs and jumping onto seats. Unlike other box seats I’ve seen in other arenas, there were only a few, somewhat small, TVs, most of which looked pretty beaten up.

There is also a menu of food offerings. If you thought NYC was expensive before, you’ve seen nothing like this: $40 for a tray of chicken nuggets and pigs-in-a-blanket, enough to serve 4 or so kids. This goes up to $99 for a tray of pasta. 6 packs of beer were $30 for Bud Lite. It was staggering. Oh, and they confiscate any food/drink you bring in, including water (we were searched at the gate).

Hey, the circus souvenirs aren’t any better: from $15 for a “program” booklet to $25 for a flashing light spinner, its easy to see where the profit comes from in the circus game… well, from this stuff and the expensive tickets. But enough carping about cash.

Anyway, its clear that Ringling has had to face the world of Cirque du Soleil who changed the circus into “performance art”, the world of animal cruelty charges at every turn (they were protesting outside of the Garden, pushing brochures into kids faces), video games, fragmented media, and the general early tweening of youth.

How did they do? Not bad. Other than that mistake of using Prodigy tunes to soundtrack an act (with no words, thankfully, and no one I was with recognized the tune), they had a very clean Broadway style of songs and “parades”. Act 1 had lots of Cirque style acrobatics, very little trapeze. A very impressive 7 motorcycles in a cage was really cool… but the younger kids didn’t care much about.

The 2nd act had the animals: no lions, no Gunther Gebel-Williams who died in 2001, but an OK tiger act and a very nice elephants act. The kids were pretty rapt for the elephants, but even I was bored with the tigers.

Kids of Sam’s age, 3-4, don’t yet see the danger in some of the feats, so the trapeze and some of the other high-flier acts didn’t hold their interests.

The clowns also didn’t have as much physical comedy as I recall. There was the standard recurring joke of the ringmaster’s hat, which is ok, but I remember seeing clowns with big shoes, all climbing out of 1 small car, and lots of broad physical comedy. There was some cutesy, but the opening act was a dog run with frisbee catching. Come on, this is NYC; I can see that on Sunday afternoons in the park. Give me some clowning!

Also, the circus benefits from being down closer to the surface. Lots of comedy was missed because the clowns scatter and play to their section. From up top, you can see all of it, but miss all the details. Also, the trapeze act was obscured by the LED light show screens, so we only saw spinning feet and hands, never the whole person. But it was a pretty comfortable way to watch the show, esp. with a bathroom right there.

So, the box was fun, but we worry: this was Sam’s first time ever in a stadium. I sure hope he doesn’t think every visit to a show will be like this! The other problem, of course, is that he wants to see elephants every day now. Thank goodness for Flickr.

So, go see the circus. Like everything else, it’s not like you remember it, and don’t expect the kids to love it all… but they parts they like, they will remember forever.

Comments? [1]

* * *


Right Mouse Click on a Mac · 04/07/2008 11:59 AM, Tech

Saw this on a mailing list, and decided to save it. Not saying I’m going to get a Mac, but we’ll see…

How to “right-mouse click” on a Mac? There are a few ways:

Comments? [1]

* * *


Super Pii Pii Brothers · 04/01/2008 02:04 PM, Trivial

Finally, the perfect use for a Wii on this first day in April.

Super Pii Pii Brothers at ThinkGeek


* * *


Sometimes, you need a quick database... · 03/28/2008 02:48 AM, Database Analysis

I was working with one of the smart guys here at Y!, and he was trying to merge together a bunch of files and run some counts; the combined file size was approach 3 million lines across 32 columns. He was using MS Access 2003 as his database, and it was groaning under the strain.

MS Access has been around for a long time, and it’s JET engine age is starting to show. Even with the upgrades to the Microsoft Desktop Engine (MSDE) (which itself was replaced by the SQL Server 2005 Express Edition and then something called the SQL Server 2005 Compact Edition), it’s still kind of screams “Look, I’m better than dBase III, so you should just be happy, sit there, and shut up”.

I decided to do some digging and see if we could find something faster but still easy to use. I also wanted it to be low impact on the system, something that could be blown away easily without messing up the box.

It turns out to be harder than it looks. There are a collection of Portable WAMPS (Portable meaning installs in one directory, suitable for a USB drive; WAMPs are Windows Apache MySql Perl/Python stacks) like XAMPP but if you just want MySQL, its hard to track just that., my usual source, was lacking this (though, as usual, I downloaded a bunch of other useful apps in portable format: great to unzip, use, blow away…)

I finally found a good one here at called PortableDB. Its a nice MySql 5 distro for Windows with simple tray-based start and stops. Biggest issue: it uses a different port from the usual MySql installs (33060 instead of 3306) so some programs get confused.

Ok, database: check… but what about access tools? Portable and free were my requirements, but not much out there to support Import, Export, and a good Sql query window.

If the tool is a “native MySQL” tool, it may let you make new databases. Java and “generic” tools may not let you do this (though they can make new tables, etc.), but it’s pretty easy to get the MySQL command line up and running and use that to make the database. You may be able to do it from within the tool’s query window, but its just safer to do this from the MySQL command line prompt.

The other gotcha is that some of these tools limit the rows returned to reduce memory consumption (it takes lots of memory to set up those cute grids). It’s worth checking the Preferences and hunting around to see if they aren’t putting some kind of LIMIT or other restriction. Sometimes you really need to get down in the data and not being able to get more than, say, 4,000 rows can cause an issue… esp. if you miss the fact that it’s your query tool stopping you; there might be nothing wrong with the data.

Some options:

Yes, MySql has its own graphic tools, but they are both hard to use and underpowered (see MySQL Query Browser for example). Yes, you could install a WAMP and use PHPMyAdmin which is an ok idea for simple stuff, but analytic queries benefit from a gui with copy/paste, sorting, multiple windows, etc. Yes, you could use MSAccess via ODBC, but that’s kind of defeating the purpose, isn’t it? Another option might be to try OpenOffice’s Base, which includes both HSQL and ODBC, giving you two options, but it suffers from trying to be too much like Access.

So, HeidiSQL seems to be the winner, esp. in the PortableApps version. But it might be worth playing with some of the others to see if they meet your needs, esp SQLYog in the Lite/Community edition.

BTW: interesting find, might be cool:
Toad for Data Analysis and Toad for Data Analysis

PS: To import CSV in HeidiSQL, you first need to make the table (which can be a pain) and then configure the Import like this:


terminated by ;
enclosed by “ (check the Optional box)
escaped by \\

terminated by \r\n
ignore 1 Lines (if your first line is a header)

If you want other ways, consider this sql statement:

load data local infile ‘uniq.csv’ into table tblUniq
fields terminated by ‘,’
enclosed by ‘”’
lines terminated by ‘\n’
(uniqName, uniqCity, uniqComments)

LOCAL means take from local filesystem. “Ignore Row “can be used to skip the header.

There is also a “wrapper” program that MySQL includes which tries to give you many steps in one command line:
mysqlimport —fields-terminated-by=”,” —fields-optionally-enclosed-by=”\”“ —lines-terminated-by=”\n” geoip /mnt/www/csv.csv -u root -p

Note that you still need to create a table ahead of time, which is a pain: mysql -e ‘CREATE TABLE tlbUniq(id INT, n VARCHAR)’ test

I am kind of amazed: there is no real free “import wizard” ala the Excel or SPSS wizard. It would be pretty easy:
Read file, print the first few lines, try to guess format of the data, llet the user correct, and then create the table and import. Yet no one has done it yet? Surprising…

Comments? [5]

* * *


Open Source Web Analytics · 03/26/2008 11:53 AM, Analysis Marketing

It’s been a slow time for open source web analysis systems. At one point, there were dozens to play with. But the movement from log files to beacon tracking, along with the move to “free” for tools like Google Analytics and MSN Gatineau have really put a crimp on growth.

However, the desire to control your own data along with customization of interface has started some people coming back around.

The best I have seen is a young project called They have spent a great amount of time on making a fully customizable front end dashboard, and they’ve leveraged their back end from the work they did on the PHPMyVisites project (yes, the open source one with the heatmap). Anyway, you can see a demo of Piwik in action at the Piwik Online Demo.

It’s got lots of the good stuff, including sparklines, smart charts, the adjustable interface, and a clean look. But its also clearly an alpha, with a paucity of segmentation and filtering options.

Is it a Google Analysis killer? Not yet. And does it deliver anything unique that the other tools don’t? Well, not quite yet, though the dashboard component process is well done. But I think there is a lot of hope here, and I encourage the advanced web analytic community to start playing with this tool more, and consider giving our help to them.

There are others as well. Brett Bittke blogged about a good list on March 20th, 2008; List of Google Analytics Alternatives. He mentions the usual, including Awstats, which is preinstalled on most every shared host these days, and is one of the more comprehensive log file packages as well as the collection of “visitor count” systems like BBclone, which don’t really get into usage and instead just show simple aggregate PVs and user counts. He also mentions lots of cheap pay systems, most of which aren’t really delivering anything beyond Google’s system.

Awstats is really out of date, and I don’t think the “visitor counters” are all that useful, so we have nowhere to go but up.

(In fact, as I’ll post about soon, I don’t believe any of the “web analytic tools” are really there yet. But once you read my post about the commercial tools, and get past calling me an idiot, I think you’ll see that in some ways, the open source area may be our best hope to truly start understanding how people are using interactive media. It’s still baking, but look for it soon.)

I will be looking more deeply into Piwik, and I encourage my readers to do the same. It’s got a ways to go, but I think as a foundation, it’s one of the better ones to use as we create the next generation of web tracking systems… and perhaps, finally, earn the name “analytics”.

(PS: If you want to read more places where Open Source is dropping the ball, check out my post on PHP and BI.)

(PPS: I got tagged for not mentioning Open Web Analytics which is basically an open source framework for adding web analytics hooks to sites. It seems more techie than it needs to be, but find out more at their site or in this pdf.)

Comments? [7]

* * *


Twitter · 03/19/2008 08:45 PM, Personal

So, I’ve added a Twitter Page which is really sort of silly. Basically, its for microblogging anything which fits into 140 characters. And you can do it from your phone.

I know, seems kinda silly to me too, but like “podcasts”, people do seem to like it.

We complain about sound bites, but we sure do love to provide ways to share them.

Anyway, if you want, feel free to “follow” me and I’ll try to pop some stuff on there at various times.


* * *


On a previous episode...

powered by Textpattern 4.0.4 (r1956)