December 20, 2011July 25, 2018 by martha

Tuesday Newsday: Data News Sites

Data journalism, news apps, open government data. In this age of Anonymous, Wikileaks, and proactive sources publishing freely available data, it is becoming critical that we have journalists and developers working together to make sense of it all and understand how it affects us.

Today there are a growing number of news outlets doing a fantastic job of both publishing data and creating interactions and visualizations to make sense of it. I wanted to showcase a handful of those sites so that people can understand why this type of journalism is important and how it works. And maybe even how to get involved if it’s of interest to you.

DATA SET PROVIDERS:

There is no shortage of people providing data sets today, the only thing to learn is how to narrow it down to the most useful one for you.

Fingal Open Data is doing a nice job in Ireland encouraging councils to open up their data. Their site http://data.fingal.ie/ provides data in CSV, XML and KML formats, and they’re currently running a competition for people to build useful apps given the available data.

Additionally in Ireland, sites like Gavin Sheridan’s The Story and John Handelaar’s Kildare Street are excellent resources for presenting data but also learning more about the types of data available in Ireland

I mentioned The Guardian’s Data Blog in a recent post about news developer blogs as a great example of community building. But clearly it is also one of the best sources in the UK for a wide range of freely available data sets. For each set of data they post, they offer a download of the data and recommend that if you do anything with it, you post it to their Flickr group. They often have additional visualizations and comparisons as well.

There are loads of additional resources for finding data, the trick is knowing how to search for the information you need. Sites like ScraperWiki (https://scraperwiki.com/), DataSift (http://datasift.com/), The World Bank (http://data.worldbank.org/), BuzzData (http://buzzdata.com/) and more sites are growing and becoming available to the public for research and data mining.

NEWS APP EXAMPLES:

It’s no coincidence that a lot of my news app examples have shown up in prior blog posts – there are some organizations who are very seriously ahead of the pack in terms of understanding how news and technology work together. On one hand I am delighted to have bright minds like these folks paving the way! On the other hand, it makes me a little depressed when I realize that Ireland is nowhere near this level of thinking.

NPR’s StateImpact (http://stateimpact.npr.org/) is a collaboration among NPR and local public radio stations in eight pilot states to examine public policy issues in depth. They provide explanatory, data-driven stories focused on how people’s lives are affected by government decisions.

The New York Times definitely has a well-respected team of interactive news developers and infographic designers. However it’s not always very easy to find them on their site. The Multimedia page is the best resource I’ve found for taking a look at cool interactive apps and images they create, but Small labs Inc has also put together a nice collection here: http://www.smallmeans.com/new-york-times-infographics/

The Chicago Tribune has some of the busiest & best news app builders in the United States today and they are constantly kicking out new, interesting information in nice visual formats. Check out their site which has apps in many areas including community, schools, business, politics and more.

ProPublica creates not only fantastic interactive news applications like Dollars for Docs (which I referenced in my Quantified Self talk on skin problems), they also create excellent reusable tools like DocDiver that allow readers to work with the ProPublica reporters.

DATA JOURNALISM RESOURCES:

To learn more about data journalism, here are a few useful sites to check out and blogs to follow:

The Data Journalism Handbook: a work-in-progress coordinated by the European Journalism Centre & the Open Knowledge Foundation launched at the Mozilla Festival in London on November 5th 2011.

Data Journalism Blog: http://www.datajournalismblog.com/

Hacks/Hackers: http://hackshackers.com A grassroots journalism organization on a mission to create a network of journalists and technologists who rethink the future of news and information.

ProPublica’s Dan Nguyen’s Scraping for Journalism: A Guide for Collecting Data http://www.propublica.org/nerds/item/doc-dollars-guides-collecting-the-data – a great introduction to how to grab data that is in maybe less-than-optimal formats.

Also from Dan Nguyen, The Bastards Book of Ruby http://ruby.bastardsbook.com/ – an introduction to programming and its practical uses for journalists, researchers, scientists, analysts and anyone else who needs to make sense of data.

At the Society of News Design conference earlier this year, I went to some excellent talks by newsroom app developers who recommended that if you are thinking of going into this area, you need to know either Ruby on Rails or Django/Python, as those seem to be the dominant requested skills for news apps. I also liked this article recently on Poynter about using Backbone to create data news apps: http://www.poynter.org/how-tos/digital-strategies/147605/how-journalists-can-use-backbone-to-create-data-driven-projects/. You don’t need to know everything, just pick one and start creating projects.

And lastly, from the perfect timing department, here’s Matt Stiles, who is a data journalist from NPR, talking very recently about best practices in building news apps:

Data Best Practices from StateImpact on Vimeo.

October 20, 2011August 11, 2018 by martha

Visualizing Áras Election, Part Two

I received some good feedback on additional things people thought might be interesting to look investigate regarding next week’s election. So today I’ve done two things: 1) Taken a brief look at media outlet perception of candidates based on published articles and 2) Re-examined the Irish twitter stream with a new sentiment engine to see how the individual candidates stack up against each other.

NEWS SOURCE PERCEPTIONS

Once again using ScraperWiki, I picked two different news sites to scrape for their election coverage. RTÉ and The Irish Times both make it a bit awkward to find all of their Áras election coverage in one place. On RTÉ, the best source was http://www.rte.ie/news/presidential-special-reports.html, but I’m not positive it’s comprehensive as it was a link I stumbled on to while digging around their site. For The Irish Times, I used a search function to pull up 100 articles containing the word “Aras.”

The best site, which I ran out of time today to include but will add it later, is unsurprisingly TheJournal.ie as they have a nice tagging system. You can simply visit http://www.thejournal.ie/topic/race-for-the-aras/ for all of their great election coverage.

Back to The Irish Times and RTÉ: using scrapers to comb through their HTML I pulled out article titles and descriptions to get a brief understanding of what tone comes through and who is talked about most. With more time, one could easily walk through all of the articles and grab and parse that text as well, but this is a basic exercise. Another tough thing about RTÉ’s coverage and what may limit me digging deeper there is that they have so much video coverage and I couldn’t seem to find any transcripts of video reports. Parsing audio into text from video reports is a whole other project!

The Irish Times:

The Irish Times election article word cloud

RTÉ

RTE election article word cloud

The data sets and visualizations are all linked on my Many Eyes page here in case you’d like to do your own visualization of the data.

IMPROVED TWITTER SENTIMENT

In my last related post, I used twitrratr to do a very simple analysis of how people in Ireland were feeling about the election. However it is a very simple application and I wanted to expand on it, using a better sentiment algorithm.

R is a statistical computing and graphics generation language and tool. R allows very interesting and complex analysis of language and data. I used two R tools to help source and evaluate the Tweets. First I used Jeff Gentry’s twitteR package which has some very easy methods for searching twitter timelines. A search for tweets related to David Norris, for example, might look something like this:

norris.tweets = searchTwitter(‘aras11 AND norris OR david OR SenDavidNorris’, n=1500)

where the words in quotes are my search terms and the n=1500 refers to how many tweets it should return. So I built queries like these to search for tweets related to the individual candidates.

The next tool I used was an “opinion lexicon” by Hu & Liu. If you’re not familiar with processing language, the easiest way to explain this is it’s a big dictionary with almost 7,000 words which are categorized as positive or negative. Words like “love” or “amazing” would be categorized as positive, and words like “hate” or “sucks” would be considered negative. Of course this doesn’t allow for sarcasm, so we have to assume that most people mean what they say. In the future maybe we’ll have to also search for a “sarcasm” hash tag and then reverse the word values!

With the opinion lexicon, we can go through all of the tweets and score them depending on whether the words in the tweet are more positive or negative.

Finally, we can plot the answers on a histogram as shown below. The diagram is a bar chart showing for each candidate, how many tweets were considered positive versus negative. We can see that Dana Scallon has relatively more negative tweets than the others, and that Michael Higgins has relatively more positive tweets than the others. Higgins also seems to have the widest variety, with tweets going up to a score of six and down to a negative five.

Histogram of Irish Candidate Sentiment on Twitter

*many thanks to Jeffrey Breen for his excellent slides on Twitter text mining and for publishing the code – very helpful!*

WHAT ELSE?

I’d like to go deeper into the actual published articles, which will take a bit more time but could provide some interesting results. I would also love to look at additional sources such as TheJournal.ie and The Irish Independent. As I have with the previous charts, I’ll continue to update these daily until the election and see if Twitter is able to make a good prediction about the final result.

After the election I’ll also do a blog post on how to create your own data visualizations from public sources with easy tools that you don’t have to be a programmer to use.

October 17, 2011August 11, 2018 by martha

Visualizing Áras Election

The Irish presidential election is just about a week away. As a non-citizen resident of Ireland, I can’t vote in this election (only local elections). But I still find it interesting so I took a look at some social media data on the topic to make some visualizations. These are not meant to be predictions, it’s just a bit of fun to see what people are thinking today in Ireland.

I am using a tool called ScraperWiki that I learned earlier this year at a Hacks and Hackers Day in Dublin. ScraperWiki lets you scrape data from various sources such as a PDF or in this case, Twitter. My scraper grabs any tweet mentioning aras, aras11 or president originating from Ireland.

JUST THE WORDS

We can use a tool like IBM’s Many Eyes to visualize the most frequently referenced words in these tweets. The visualization below, embedded from Many Eyes, shows that Norris and Gallagher are probably the two most discussed politicians on Twitter. You can right click on the visualization to alter it, remove certain words (I removed things like “RT” and “QUOT” and “ARAS11” as they weren’t relevant), change colours, etc.

*NOTE: Many Eyes is a Java tool, so you will need Java to interact with the data. If you can’t view the visualizations, please scroll to the bottom where I have screenshots of the data instead*

WHAT ABOUT ASSOCIATIONS?

More interesting than the individual words themselves, to me, are the associations they have. In other words, is one candidate’s name mentioned frequently in the context of other certain words or phrases?

The Customized Word Tree, another tool from Many Eyes, allows you to upload a text and then enter specific words to find other terms associated with it. To use this interactive tool, simply type in a name like “Gallagher”, “Norris”, “Dana”, etc. into the Search textbox & hit return. You’ll see a visualization of words and phrases most frequently associated with that candidate.

*NOTE: Many Eyes is a Java tool, so you will need Java to interact with the data. If you can’t view the visualizations, please scroll to the bottom where I have screenshots of the data instead*

SENTIMENT

Does anyone care about sentiment analysis anymore? Sentiment analysis is trying to understand the general feeling, positive or negative, from a group given a topic. So if you did sentiment analysis on Twitter for the term “taxes”, you’d probably find most people associate that with negative feelings, frown emoticons, and an overall negative sentiment. Unless of course the government had announced huge tax refunds for everyone, in which case it would likely be overwhelmingly positive.

twitrratr is an example of a tool that does sentiment analysis given a topic. It’s as simple to use as Twitter search, but the results in this case aren’t incredibly useful. You can check the sentiment yourself easily by clicking here: http://twitrratr.com/search/aras11.

HOW TO CREATE YOUR OWN

ScraperWiki is great because you can use a variety of programming languages and it has support for lots of different sources including PDFs which are notoriously hard to parse.

I forked a basic Twitter scraper that looks for tweets containing keywords. You can see my scraper here: https://scraperwiki.com/scrapers/aras_election_data/. The Twitter search API lets you use regular expressions, so I edited the keyword to be ‘aras OR aras11 OR president’. Searching for president could definitely bring up irrelevant tweets for this purpose, so I also added a geolocation query. The Twitter search API lets you use a latitude & longitude followed by a radius to find tweets in a particular area. I added some very simple Python code to the scraper to allow it to handle geolocation queries.

As you’re developing your scraper, you can run it every time you change something to make sure you are getting the results you expect. Once you’ve finished, you can schedule it to run daily, weekly, etc. If you get stuck, the ScraperWiki community is a great group of people, they have a very active Google Group and growing documentation.

Once you have the data you need, you can export it as a SQLite database or a CSV file. There are plenty of tools you can use with this data. Many Eyes is a good one to start with as it’s very user friendly. If you’re into programming, there are many good JavaScript libraries and other tools you can use to manipulate the data. Just search online for things like “data visualization tools.”

MORE ON THIS

My scraper runs once a day, so I’ll be updating the interactive charts daily from now until October 26th when the election is held. If there’s other information you think would be useful or interesting to look at, related to the candidates or the upcoming election, please leave a comment and I’ll take a look.

SCREENSHOTS FOR THOSE WHO CAN’T VIEW MANY EYES

Word Analysis:

Candidate Association Examples:

October 11, 2011August 11, 2018 by martha

SND STL Review, Part Two

SND STL Logo

CONFERENCE DAY 2

The second day of SND STL had me overwhelmed. I listened to so many great speakers the previous day, learned so much, saw so many excellent demos. And now here I was with another great line-up of sessions to choose from.

Tweet Cross-Platform Editing – Teresa Schmedding is the president of the American Copy Editors Society and also an assistant managing editor at the Daily Herald in Chicago. Teresa talked about trying to hold the line between editorial quality and actually getting stuff out the door. Her stories of pet peeves of writers and editors she’s dealt with were highly entertaining. It’s hard to know what’s worth fighting for. You will have readers who will write and complain about certain edits so you have to know what your standards are.

As someone who finds it near impossible to read blogs, websites, even Twitter feeds, and of course books and print media, with bad grammar and spelling, I sympathize with these tough calls. I know carelessness can lose you readers.

Teresa’s top recommendations were three things:
1) Focus on grammar, it’s the most important
2) Details: make sure you’ve got ‘em and they’re consistent
and
3) Structure: make sure it’s not confusing for the reader

Teresa recommended not to spend time on some of your old and time-consuming style rules, certain old “newspaper” rules, and your boss’s pet peeves. Ask people, your readers, what they value so you can ensure you’re spending time on what’s most important to your readers. And if you’re thinking about a pay wall, people will not pay for error-filled copy whether they are grammatical, spelling, or fact errors.

Tweet Designing a Responsive News Website – Everyone working on the web at this point has to have seen the Boston Globe redesign at http://bostonglobe.com. Miranda Mulligan and Mat Marquis are two of the main folks behind the redesign. They talked very honestly about what went into the redesign, what was hard, what was valuable and what pieces they’re still working on. They described HTML5 & Responsive Design and which pieces they wanted to incorporate in their new site. They also showed us the grids they ended up using and how they worked with things like Scott Jehl’s Responsive Images script, media queries and other technology to make it work right.

Challenges they came across included the Boston Globe masthead (very complicated & went through many revisions), third party integration for things like video and advertising, making HTML5 crossword puzzles (this piece is currently in beta), saving offline stories, and interactive information graphics in responsive sites.

Also interesting to me was that they chose a new CMS and went with http://www.eidosmedia.com which is also used by the Denver Post, the New York Post and the Washington Post. It will take two years to implement and customize, print products will launch on it next year.

Lastly they covered the importance of using analytics and allowing them to inform future decisions such as grid widths. Excellent and candid talk about the real pain and equally real rewards of going through this process.

Gamestorming with Dave Gray – I read Gamestorming earlier this year and found it to be a very useful book for developing ideas and working on teams. Dave gave some examples from his book and how companies like Starbucks are listening to customers and changing their strategies for the better.

Dave pointed out that the question is no longer “What’s in tomorrow’s paper”, it’s now “What IS tomorrow’s paper?” Companies who approach it this way will get it right faster than those who are clinging tightly to old models that are going to slowly fade away.

The paradox of discovery is that you might not find what you’re looking for, but if you’re not looking for something then you won’t find anything. If you’re looking for the ROI before you start anything, you’ll never build anything. You don’t have to be smart, you just have to do things.

Dave mentioned if you think you have a problem with overthinking it, you should go to http://overthinkingit.com. For his book and more on Gamestorming, go to http://gogamestorm.com.

Tweet How to Make News Apps in your Newsroom – Several times on this blog I’ve mentioned the fact that trying lots of different things and being able to work news, magazines and media more like an agile start-up is a great way to get ahead of the competition. Brian Boyer from The Chicago Tribune and Scott Klein from ProPublica talked about news apps which are software. This is a new area, so these aren’t people in your company who can fix your computer or get the server back online, they’re people who can build newly capable applications to visualize stories or interact with users.

Apps that fall into this category include things like “Dollars for Docs” from ProPublica, which points out doctors who have taken money or other compensation from pharmaceutical companies. The speakers pointed out we need “Demos not memos.” In other words, we need more people in the field who can help to visualize and display and build these applications.

If you’re interested in working in this area, good places to start are things like Hacks & Hackers, NICAR-L mailing list, a language like Python or Ruby, a database like MySQL or PostgreSQL, and informational, how-to sites for web designers/developers like http://developer.mozilla.com and http://alistapart.com. Lots of other helpful links are here on Brian’s pinboard site: http://bit.ly/sndnerd.

Tweet Finding the Web Designer Within – These two guys from Upstatement had amazing slides that they published ahead of time so you could follow along. The slides are great because they point out that a lot of what news designers have been trained to do are very useful skills for web design. Things like grids, typography and being able to organize volumes of information are all skills that can transfer to web design.

As with the previous speakers, they mention that it’s important to teach yourself to code and recommended Coda. They design in the browser but still use InDesign occasionally.

You can find the Upstatement slides from this talk here: http://upstatement.com/sndstl/

Tweet Adios, Arial! New Tools for Taking Beautiful Typography from Print to Digital – Typography is so important for readability on the web, and Alan Tam, Sam Berlow and Danny DeBelius gave a great talk and examples on why you need to test every scenario, why the results can change and some technology to use to help. Sam recommended tools like the web font preview on http://www.fonts.com, the fontswapper on http://www.webtype.com and http://fontfonter.com to try web fonts on any website.

Great fonts can render totally differently on different operating systems, different devices, different browsers, so it’s important to test the fonts you’ve selected everywhere. It’s also important to make sure you’re using complementary fonts, which is probably hard to eyeball unless you’re a very seasoned web font user.

Tweet Closing Keynote with Robin Sloan and Matt Thompson – This presentation was definitely one you did not want to be late for. The session was live-streamed, and below is the video. Robin and Matt are amazing speakers and it’s definitely worth watching their take on what the future of reporting a natural disaster might be like with their video called The Storm Collection and the brilliant talk that follows:

You can view the slides from Robin and Matt’s talk here: http://snarkmarket.com/storm/

END OF DAY 2

I’m very used to attending technology conferences with topics I’m more or less familiar with. It’s been a long time since I have been to an event where I felt that every talk I walked away from had new ideas and new things for me to learn. As I said in the beginning of this post, it’s overwhelming. Although I’ve always been a newspapers and magazine junkie, that’s not what I studied in school. Without a journalism background, I felt out of my element. It’s uncomfortable to be in that position, but I realize we probably don’t do that enough to ourselves.

SND STL was incredibly well put together and I am so grateful to the organizers for all their hard work and effort. I will definitely be attending SND again in the future, hope to see everyone next year in Cleveland.

October 6, 2011August 11, 2018 by martha

SND STL Review, Part One

SND STL Logo When your hometown is in the Midwest, you generally don’t expect to get the opportunity to fly there often for amazing, cutting-edge, technology and designy events. I was surprised that SND, the Society for News Design, was having their annual event in St. Louis, but even more surprised when I looked at the line-up and saw that it was quite a visionary event, with speakers from Zite, Font Bureau and The Boston Globe and incredible folks like Josh Clark, Charles Apple, Dave Gray, and Robin Sloan and Matt Thompson giving talks and speaking on panels. Every time I looked at the schedule I got excited. Plus the SND STL website was one of the best conference sites I’ve ever seen.

So here’s a brief overview of the event:

Tweet DATA VISUALIZATION WORKSHOP

I had signed up for a pre-conference workshop on data visualization. Having done a graduate course last year in designing and creating data visualizations, I wasn’t sure how much of it would be duplicate, but I figured I could use a refresher. Plus the workshop speakers were from The New York Times, Duke University and Investigative Reporters and Editors, so these were people with some unbelievable backgrounds and experiences. The workshop was really good, and I walked away with some very useful resources. We spent time in Excel of course, but also Google Fusion Tables & Google Refine. We took a look at several open source tools I hadn’t heard of to better parse and massage data. Lastly we looked at some public projects from US news websites and their blog entries or overviews of how they built their solution.

Here is a list of some of the resources we walked through, I hadn’t heard of several of them so it was really helpful for me.

http://sparklines-bitworking.appspot.com – Sparkline graphic generator
http://addressextract.appspot.com/ – Tool for fixing addresses
http://pinboard.in/u:dwillis – pinboard site of Derek Willis (one of the instructors, works at NYT), lots of useful links
http://mapbox.com/tilemill – Open source map design software
http://foolabs.com/xpdf – cool tool for extracting from PDFs
http://raphaeljs.com/ — raphael.js — nice small charting svg library in js, used to make the NYT voting result maps
http://elections.nytimes.com/2012/campaign-finance#canda=barack-obama&candb=barack-obama-2007 – slick example of using some svg charts live on NYT site
http://documentcloud.github.com/backbone/ — backbone.js – allows you to persist data & update it in real time based on user interaction
http://shancarter.com/data_converter/ — Mr. Data converter: csv or tab-delimited data & export as various formats
http://github.com/FlowingMedia/TimeFlow/wiki – comes with some nice examples to show you how to use it
http://propublica.github.com/timeline-setter/doc/twitter-demo.html – Cool Twitter timeline setter demo
http://pinboard.in/u:dwillis/t:snd11/ – List from Instructor Derek Willis of links used in class
http://freevisualtools.wikispaces.com/Some+FAVES+(shown+at+SND+STL.) – Links used in another conference session which are relevant here too

Some awesome data & tools blogs to follow from news sites are:
Propublica Nerds Blog – http://www.propublica.org/nerds
The Guardian Data Blog – http://www.guardian.co.uk/news/datablog
NY Times’ new beta620 labs site – http://beta620.nytimes.com/
Madison.com Labs blog: http://labs.madison.com/blog
LA Times Data Desk: http://projects.latimes.com/index/
The Texas Tribune Data: http://www.texastribune.org/library/data/
Data Journalism blog: http://www.datajournalismblog.com/blog/

(note: these links all work & are correct as of October 6, 2011)

Tweet CONFERENCE DAY 1

The first day of the conference started with a latte and pumpkin gooey butter cake (a St. Louis specialty) from Park Avenue Coffee around the corner from the venue. One thing I learned: journalism conferences do not have the abundance of coffee and baristas that I’m used to with tech conferences!

Tweet Mobile & Tablet Research Roundup – This was an overview of tablet and mobile research by Roger Fidler & Regina McCombs. Some interesting stats from thinkMobile & Google research on how people are using smart phones

89% use it to stay connected
82% use it to read news and research
75% use it to navigate
65% use it to keep entertained
45% use it for management and planning

Their research also showed that GPS is one of the most important features for smart phone users, and good apps should incorporate it appropriately. Social media comes in second in terms of feature importance.

For news sites, having usable mobile capability typically adds 10-20% to traffic. However the number is greatly affected in disaster or breaking news scenarios. As an example, CNN.com had 13.9 million mobile page views during the Japanese tsunami this year.

Tablet sessions are long, 58% of people use it for an hour at a time, 30% use it for > 2 hours at a time. Daily.

What are they doing on the tablets? Using engaging, longer form, higher production value applications like games, magazines and video. Newspapers like the Daily Telegraph found tablet owners are not generally interested in breaking news, more interested in more detail and interactive pieces.

Regina posted notes from the research & sources here: http://delicious.com/reginajmc/sndstl

Tweet Fun101 — A strange title for a talk at a news design conference, but Fun101 with Tim Harrower was one of the best sessions I went to. Tim had amazing ideas on how to engage readers, keep them interested and get them to have fun. Which is especially relevant today with so much depressing news. Ideas Tim mentioned included things like predicting awards shows ahead of time, fun quizzes, games to tell news stories, using comics to explain difficult or complex stories, and a lot more. I’m not seeing his slides online, but you should definitely check out his website for excellent and thought-provoking ideas and content: http://www.timharrower.com/

Tweet Designing Personalized Tablet News – This was a panel with Bobby Ghoshal from Flud, Mark Johnson from Zite & Joey Marburger from Trove. It was a good panel where each aggregator talked about their decisions on multiple platforms, how much setup to give the user (i.e., do you seed them with feeds, let them choose, etc.), how they allow users to share, revenue streams and more. My takeaway from this was that these are in a dangerous spot as they rely a great deal on the lack of pay walls or a system to integrate with news sites. The aggregators don’t want to do a ton of work for each site they allow their users to access so the tougher a news site’s pay wall, the more likely the aggregators won’t include it. Having just seen the fantastic film Page One: Inside the New York Times, I fear what could happen if everyone stops paying for news or thinks they’re entitled to quality reporting for free forever. So while the aggregators do provide eyeballs, it seems there needs to be money changing hands somewhere for people to be able to use these apps. All three apps are currently free, but it doesn’t seem to be a sustainable model.

Tweet Keynote Speaker: Rob King – Rob King, VP & Editor-in-Chief of ESPN Digital Media gave a fantastic talk on dealing with change in your personal and professional life. My favourite quote: “It’s going to work out, you just don’t know how yet.” Brilliant.

Tweet Designing the Magazine and Issue-based Tablet Experience – This talk was given by Mike Schmidt from The Daily, Claus Enevoldsen from Next Issue Media and Robert Newman from Reader’s Digest. Very good session on the tools they use, how they spend their time and production flow. I was terrified to learn that at The Daily, there are 50 people on just the design team, and that they have no automatic templates so every day they handcraft over 100 pages. Then they do it again in portrait mode. Yikes.

Tweet Touching News: The New Rules of Tablet Media – This session was one of the ones that made me sign up for this conference. I read Josh Clark’s book Tapworthy earlier this year and loved it. Josh had some of the best slides of the event, but he also automated his Twitter account to tweet additional interesting blurbs about his talk as he was speaking. Magic! Josh had excellent UI tips, usability and interaction examples, and was basically all around brilliant. Excellent speaker – go listen to him if he’s speaking near you any time soon. His beautiful slides are here: http://globalmoxie.com/jhc/prez/touching-news.pdf

Tweet Building in HTML5 & Bypassing Native Apps – I loved this session, not just a little because one of the developers from The Onion was speaking. Alan Herzberger from The Oklahoman & Michael Wnuk from The Onion gave very honest overviews of why they chose to build with HTML5 instead of iOS and their internal processes that happened for design, development, CMS modification, etc. Very interesting and candid discussion about the pain they went through, what works and what doesn’t.

Tweet Visual Conceit: The Secret Ingredient of the Secret Ingredient – Adonis Durado has a tough job. As Design Director of two newspapers in Oman, he works with designs and layouts in both English and Arabic and has to deal with a very reserved audience where showing skin to get more readers is not an option. Adonis talked about creating pages and layouts that make people think and surprise them. He showed a lot of examples of his papers’ compelling designs and “Wow factor” creations. He cited hiring a diverse team with different backgrounds and voices as one of the defining factors of his successful transformation of the papers from unknown to award-winning. His hand-out from the talk is here: http://www.scribd.com/doc/66821299/Visual-Conceit-Handout

END OF DAY 1

There were so many great talks, every session was a tough decision but I was quite happy with how much I learned. Links to other speakers plus slides and blogs can be found here: http://sndstl.com/2011/09/snd-stl-take-aways/

August 27, 2011August 17, 2018 by martha

Fun With Data & Goal-Setting

I’m working on a small project which involves personal data tracking, so I’ve been doing a bit of user research on people who set goals and track their progress towards them.

43Things is a popular and easy-to-use site for setting goals and recording your progress, and I’ve enjoyed using it in the past myself. I figured they might have some useful information on what types of goals people are setting and tracking today as well as some predictors for success.

On their “Zeitgeist” page, 43Things aggregates some interesting lists from member data such as all-time most popular goals (“lose weight” is the top goal by far), new goals (“Become the next Frank Sinatra” was on there as I was writing this), popular goals today, and more.

I took the list of top 100 all-time goals and list of top 100 achieved goals and merged them to see where the overlaps are, where the gaps are, and if there is a correlation between popular goals set and popular goals achieved. I found some interesting things.

THINGS PEOPLE ACHIEVE:

If you regularly set personal or professional goals for yourself, it won’t surprise you to learn that the goals with the highest number of achievers are generally measurable. It’s probable that no one marked off having achieved “Get in shape” or “Be more confident” because they’re hard to quantify. “Lose weight” is a bad goal because unless you give yourself a bit more guidance, it is difficult to say when it’s accomplished. “Lose ten pounds in three months by decreasing my soda and snacks intake and working out three times a week” is definitely more wordy, but it’s also more concrete and has a higher likelihood of success.

I won’t try to argue if the top achieved goal, “Fall in love”, is quantifiable, but most of the top achieved goals are, in fact, very measurable.

THINGS PEOPLE IDEALLY WANT TO DO BUT DON’T:

The next interesting thing is the gaps where lots of people are setting goals but no one seems to be achieving them. Some of these are, as above, not so measurable. But others are more along the lines of “things I’d like to say I did, but I don’t have the time or the passion.” There are quite a few language learning goals in this category, as well as things that require a large commitment of time such as running a marathon, traveling the world or writing a novel.

THINGS PEOPLE DO WITHOUT EXPLICITLY SETTING A GOAL:

Have you ever written a to-do list, and then thrown an item or two on there that you’ve already finished? Just to make you have some feeling of accomplishment? I have, especially if the day’s task list looks endless. The last interesting category I noticed seemed to be these things. Things people did without realizing it was a goal they wanted to achieve. Graduating from school, getting a passport, donating blood, doing ten full push-ups all fall into this category. They’re often not the types of goals people think they need to be strategic to achieve, but that’s not to say they’re any easier than the other goals.

CREATING THE CHARTS:

I used a JavaScript charting library called Highcharts. Highcharts is incredibly simple to get up and running, and they have a larger variety of chart types than many other charting packages, including stacked bar charts which I wanted to use to show some of the gaps in certain goals. 43Things actually has an API you can use to pull this type of information dynamically, so it would be possible to create a dynamic version of these goal numbers quite easily to do something interesting like track increases or decreases of certain goals over the course of time or with respect to current events (e.g., there are probably more people with the goals of “Get out of debt” and “Pay off mortgage” today versus five years ago!).

[UPDATE-30/8/2011]BLOG POST FROM 43THINGS:

The creators of 43Things.com, The Robot Co-op, featured my blog post on their excellent blog about 43Things & the other projects they build. You can read the post here: http://blog.robotcoop.com/2011/08/30/fun-with-43-things-data/. Thanks for taking a look, Robot Co-op, and keep up the great work!

[UPDATE-27/8/2011]ALTERNATE VIEWING OF INFORMATION:

It was suggested that since Highcharts doesn’t work in some mobile browsers, I include an alternate form of viewing the data, so I’ve added the images below in case that helps. Thanks for the great suggestion.

Martha Rotter

Just another nerd

Tag / datavisualization