The Evolving Newsroom: Q&A with Conrad Quilty-Harper

The Evolving Newsroom is a series of Q&As with important names in the data journalism field, discussing how the newsroom is evolving to better incorporate data and data-driven journalism. To start things off, I’ve talked to Conrad Quilty-Harper, who is the Interactive News Editor for the Telegraph.

Ændrew Rininsland: You’d written for Engadget before, what was it like going from a technology website to a traditional newspaper?

Conrad Quilty-Harper: Well, I’d worked for Engadget back in 2007, so quite awhile ago. I was working for a lot of blogs, not just Engadget, and then I worked for Mahalo.com, which is an Internet startup.

In terms of differences between those organisations, there isn’t much difference — they’re all very good publications, chasing news and doing it better than other publications. I’ve predominantly worked for online at the Telegraph, so essentially I’ve got the additional thing which is print; we’re a newspaper, and they’re very different beasts, online and print. But in terms of finding a good story, presenting it in an accurate way, there’s difference in workflows and tools we use, but there’s no essential difference in story sense and treating stories and sources — those are pretty universal skills.

Q: Please describe your day, beginning to end.

A: I have kind of a split in types of work that I produce. One day I might be working on one to three one-off graphics, whether that’s charts or tables or some kind of interactive feature that represents the day’s news. We’ll produce a kind of quick-turnaround graphic that will go and amplify those stories, pick out a new angle in it that you can’t tell very well with words, pictures or videos, and is better told in an infographic sense. So there’s that, which is kind of the core activity; on top of that there’s more in-depth projects that might be on a week-long basis. [Someone] was working on a cycle map, so he came to us with this data and we found a story in it and we thought “How could we better represent it?” and that was something we worked on for a week or two before it went live. And then, on top of that, there’s even longer-term projects, which starts to get into the data journalism territory of the Wikileaks project, investigations into government spending. Those are very in-depth, month-long projects. It’s kind of a mix of those three time scales, and depending on what’s in the news, what news you’ve got or what sources you’re working with, then you’ll adapt based on those three things.

In terms of the day-to-day, I’ll do some forward planning looking at the diary for government calendars and what data releases are coming out, what will be on the agenda for tomorrow, what will be on the agenda for the next week, thinking how we can prepare for that. A big part is planning and making sure you know what’s going to happen, or trying to predict what’s going to happen. First thing in the morning you generally think about what graphics we’ve got from the day before that we can finish off before our 12 o’clock conference, and then at 12 o’clock conference you look to all the section editors and the editors themselves and think of more ideas for later on in the day and going into tomorrow. I work with a couple of graphic designers, a dedicated developer and another journalist, and we’ll discuss what we’re going to be working on, or another journalist will provide some data, and we’ll discuss that with the developer and the designer. The designer will tell us how it should work and then the developer will code it, and we’ll work with the developer throughout the process of how it’s going to look. Generally we sit four or five people discussing what’s coming up, what’s happening now, and how can we adapt our tools to fit in order to make a great graphic or advance some kind of story somewhere using data.

Q: How big is your team?

I’m working with a team of four other people directly, which is not at full capacity yet but will be next month. We will be doing the day-to-day stuff, making sure to get the day-to-day graphics up and making sure we have longer term projects to show off our work and working on longer-term investigations and collecting data. I’ve worked with the Lobby team quite closely, there’s about half a dozen journalists I’ve worked with on investigations for front page stories, interactives to go with front-page stories — our team works across the newsroom as well. In a sense, we provide some service to other areas, but I like it better if we’re presented with ideas and we choose the best idea, run with it and turn it into something that suits our medium. We’re not daily newspaper reporters, we don’t run out into the street and find out this information — we will generally digest information and write the right tool that goes with it.

Q: It seems you mainly do online stuff. Do you ever interact with the print side of it?

A: I’ve done many newspaper stories that are newspaper stories first and foremost, traditional scoops. They’ll go up online as well, and vice-versa. There’s no distinction between the two; if there’s a good story in print, it’ll be a good story online, and generally that means you’ll get a good graphic out of it. There’s no real distinction in terms of the medium.

Q: In what ways do your interactions with the news editor and the web editor differentiate?

A: It’s like any other newsroom, really; I’ll come up with a pitch for a story and will present it to a specialist or news editor, and they will make a judgement on it, and that will determine where it goes. It’s no different than any type of story, picture gallery, interview or whatever.

Q: Would you mind describing that workflow a little bit, how it goes from writing to the web?

A: We have a CMS, and I will write the story offline, share it with lawyers if need be, share it with an editor, share it with my team. I might not do all of that, I don’t always show stories to the lawyers or editors; sometimes I’m asked to do stories by editors, so I’ll write with them and work on graphics because of a request, but then I’ll just copy and paste it into the CMS, write the headlines, check that all the SEO terms are done correctly, put the graphic on the article; it’s just like any blog or WordPress. We all self-publish — everyone at the Telegraph self-publishes their work.

Q: So it’s very much a web-first workflow, in that it’s driven to having stories online first.

A: Yes; if it’s a breaking news story we’ll tend to put it online first, because we’re trying to keep ahead of everyone else. Things will get held back for the paper, but those are generally exclusives. If we’re doing something that’s exclusive to our newspaper, we’ll generally let the newspaper announce it and then it will go online, though that’s not a rule. But those decisions are a bit above my pay grade, I’m not one of those editors.

Q: How do other news writers perceive of your work?

A: They generally get lots of benefit out of our work. Our graphics are generally highly supplementary to their stories, you want a good interactive to tell your story better. I’ve only had positive reactions from journalists at the Telegraph to my work.

Q: But do you feel that other writers understand the data-driven aspect of it, and how data feeds into finding stories?

A: Yes, data journalism is not a new thing, reporters have always worked with data and information that’s in a spreadsheet format or a list. If you go back through out archives, we’ve always been working with numbers and data journalism’s not a new thing. The better word I prefer is “data-driven journalism,” so what you mean is I have a data set and I’m trying to find a story from it — the data will be the first thing that brings you to that information. Maybe you find that data and you find a story, but most of the time you’re led to that data and then you’ll find a story in that data. Having interactive graphics and having a team dedicated to processing large sets of tricky-to-process data is something that I’ve always had thanks for — or need for more of — from other reporters. All you need to do is show how easy you can get a scanned PDF into an spreadsheet and write your headline figures — [after the] one time you do that with a reporter, they get it.

Q: In the time you’ve been at the Telegraph, how has its newsroom changed, in relation to new technology or the open data movement? Do you perceive of there being any changes? Is it continual evolutionary process?

A: We’ve continually updated our tools and technology. We’re quite far ahead from where we were 18 months ago, in terms of tools we have available to ourselves and other reporters, in terms of charts and tables and processing data — but we haven’t stopped, we’re continually pushing forward with new tools and new techniques.

Q: In what ways have these new tools informed the reporting that you do?

A: My favourite little piece of software is PDF2XL, which allows you to take scans and turn them into Excel. It’s quite a simple thing if you put it like that, but actually, if you’re under deadline and are trying to get some information up quickly, this is an invaluable tool. It’s something I didn’t have a year ago and it’s enabled us to do a page 4 spread in the newspaper on refused honours — people who have refused honours. That was released by the cabinet office and was in a PDF they’d released with horrible formatting, badly scanned, scratchy image. In half an hour, we were able to take that and put it into a format we were able to put into our CMS for online and make an interactive table. You can search the names, you can see Lucien Freud, you can see Roald Dahl just by typing in “Dahl.” That was one of the key; I’d previously not seen any technology that could do it, but this software does it all. We also have a server that enables us to run PHP code… We’ve got databases now that we didn’t have before, that enable us to run bigger infographics and run our services — our tools and generators. That’s all new stuff.

Q: What’s your favourite part of the job?

A: I like all of it — I enjoy working on groundbreaking stories that people want to read, and also presenting complex information in an easy-to-understand manner. I like the new angles of it — for instance, with budgets, one of my stories was a council credit card spending scandal. They’d spent a hundred million on credit cards, and you can see the kind of colour in the story, but what we were able to do using these tools was take all this data from 200 councils and publish it all online for everyone to see. Which meant you had not only did you have your big impact story, but you also had 200 individual stories that people in their local area could examine and take their local politicians to account for their spending, or even congratulate them for not spending that. The sense of transparency enabled by doing large datasets and making it very contextual for people living in their local areas is something I really get excited about. It’s not only doing a good newspaper story, a good online story — it’s being able to do extra stuff and push it forward a bit further.

Q: With that in mind, what’s your take on opening up the data behind stories?

A: I do it every story, every story you can access our data. We never hide our data. Mainly because, why would we hide it? We want people to read our stuff, why would we hide our information? If people want to know the workings behind our stuff, we’ll always release it. We’ll always release the methodology, so people can hold us to account as well. If we want to hold governments to account and ask them to be more transparent, I don’t see why we shouldn’t be more transparent ourselves.

Q: Is there any instance where somebody’s taken some of your data and done something really impressive with it?

A: One of my favourite examples was the other way around — the Guardian had worked with an academic on the new boundary changes released by the boundary commission as a draft proposal. We were really frustrated, because the boundary commission had not released any interactive, reusable maps — they had released 600 PDFs with individual images of maps, that we couldn’t show to our audience as “This is your local area, this is how your boundaries have changed.” So the Guardian contacted an academic in Sheffield, who managed to figure out how to use the table data and connect that to ward-level areas, and they created an interactive map and put it side-by-side. So what we did was take their data — which we’re allowed to do under the Creative Commons license they release their data under — and we improved on it by linking the two maps together. You had the old map and the new map, and when you moved one map they moved together, when you zoomed in, they both zoomed in. So it was like two sheets of paper together on a desk; that kind of intuitiveness. And the Guardian actually borrowed our design back — in terms of collaboration, that’s the best example of what two newspapers are capable of doing. They sourced the data, we improved on the interaction and provided our input, and they improved their design likewise. There are other examples of going back and forth — we’ve used Guardian data and provided a link back and so on. It’s quite a mutually-beneficially arrangement; if you release the data transparently and have a mind of “Let’s improve this together,” you can do good things.

Q: Given the nature of sharing data and collaborative data, how would you describe your relation with other newsroom data teams?

A: Friendly! (laughs) We constantly take inspiration from other newsrooms. The New York Times is leading the field in this space and they have a massive team of 20-plus people, but we also try to take inspiration from the HTML5 movement as well. There’s quite a big culture of people out there who are just experimenting with new ways of displaying information, and new, open source tools you can use. We use a visual library I think an intern wrote somewhere in Chicago, and it’s where you take map boundaries and make them interactive, so you can click on them and change data. We use that tool and provide help to them, they provide help to us on how to update it and make it work. You’ve got Document Cloud, we’ve given some bug reports to them… There’s a whole bunch of tools out there and different bodies, making the harder tasks easier.

Q: How do you see the current state of open data within the UK?

A: It’s been good progress so far, and I’m really happy to see the kind of progress, and I’ve written about how happy I am with the open data agenda and how pretty much every bit of it has been reached on some level. I’m not happy with certain bits — I wrote an article about how crap the crime mapping has been — but it’s slowly improving, and there’s always more they can release. We have constantly done stories about new data sets that are not on the agenda and there’s always more. There’s never going to be enough. (laughs) The government is running consultations about it and we’ve had a say and given our two cents, but part of our job is to put pressure on the government and different bodies to release this data. You can put it in a way that helps both sides — “If you release this data it helps you look better, you’re more transparent” — and we can get a good graphic out of it, present information to the public and allow other developer to reuse it. It’s all mutually beneficial. Progress has been good so far, but there’s a long way to go, particularly in the mapping government areas.

Q: What’s the most frustrating aspect of your job?

A: Scanned PDFs, or data provided on the back of a piece of paper. Part of my job is about identifying areas that are bottlenecks in terms of data and how can we get access to this data in a structured format so we can create graphics around it or automate activity that people are manually doing. I don’t really find it frustrating, it’s part of my job. Part of my job is to reduce the frustration and do innovative things that were very frustrating.

Q: How do you see the idea of data-driven journalism evolving?

A: We’ve experimented with live data in the past — we did an interactive map of the royal wedding that had people tweeting experiences from the event as it was happening. We’d like to do more in that field. Interactivity in terms of people providing data to us, we’ve been trying these quizzes — we recently did a very successful one on Alzheimer’s, where you could put down if you suspect your relative had started suffering from Alzheimer’s and could put symptoms in, which were provided by a medical body. It had an interactive element; it’s that kind of stuff that interacts with social networks, we’re putting our data from our stories in front of people in a way that’s understandable and readable, making them react to it, and making them take action based on it. ProPublica did an experiment, linking one of their stories about schools to Facebook, so you could see your local area and share stories about your local school. That’s a really agenda-setting kind of experiment and I think we’ll see more of that.

Q: How does social media — particularly Facebook and Twitter — inform the workflow of your day?

A: Very much so. On a very simple level, if people are having technical difficulties with our applications and infographics, we will usually find out from them using Twitter and we can then send them a message saying we’ve fixed it. But also, taking the council credit card example again, that caused a dozen different stories in different areas, so you can track on social media how people are reacting to your story and see follow-up angles. People asking questions about your data or raising an interesting angle you might not have thought of, or put some context to it that makes it a better story — you can always track that. It’s very useful as sentiment tracking for what people are thinking about your work. Also, with the quiz example, we’ve done some quizzes where you can share what’s your virtual diet and how addicted to gadgets you are. At the end, you get a score, saying you’re addicted to gadgets or you don’t use gadgets very much, and users could share that on Twitter and compare with their friends. We’ve had really good success with that and people like those sorts of things. News games using social media might be something we explore as well.