Jun 12, 2014

The Efficient Analyst (part three): No more waiting

Nothing kills analyst productivity like the waiting game. In a world with big demands and tight deadlines, hitting roadblocks to progress can result in huge frustrations and last-minute scrambling. Where does this waiting time come from? Quite simply, analysts must wait for data if they can't get it on their own when they need it. In part three of this series, we'll review the causes of analyst wait time and offer suggestions on how to fix it.

The drivers of wait time

Analyst wait time comes from two sources: humans and technology. On the human side, data bureaucracies or process bottlenecks can make analysts dependent on the availability of other data-capable colleagues, which can burden specific individuals and create internal strife. On the technology side, bottlenecks are caused by a lack of accessibility, such as permission restrictions. hard-to-use interfaces, or complex ETL processes that do not occur frequently enough to meet the needs of data users.

The human side

Process bottlenecks arise out of labor specialization. Perhaps there is a member of your team (let's call him or her "Data Person") who has been entrusted with managing all or some of your organization's data. Data Person might have the best data skills, or the volume of data might seem to require a point person to maintain it.  

It's important to note that Data Person is not JUST a data geek, IT professional or programmer. They are THE person (or persons) on whom the organization depends to get, use, filter or generally utilize data. There are some obvious problems with this. First, if Data Person is sick, on vacation or leaves the company, analysts don't have access to the data they need (or will waste time trying to figure it out on their own). Second, if Data Person is swamped by requests and new data updates, bottlenecks can cause analyst work to be delayed or create stress among the team (including and especially Data Person).

Good data persons try to document processes so that their absence or busyness does not slow things down. While documentation is nice, it does not solve the greater issue: the skill gap between Data Person and the average data user. 

The technology side

Challenges arise when technology creates a barrier to data access. This can be by design (access is restricted on purpose) or due to structural complexity (getting the data is simply too hard for the average data user). Either way, lack of access almost guarantees data bottlenecks. Even in cases where there are no permission restrictions on data access, hard-to-use data structures still mean that Data Person must query or extract the data for the user.

Reduce waiting time by closing the gap

Closing the gap that causes wait time must be approached from both angles. Technology must be used to create a system where data is easy to access and meets ever-changing analyst needs. This can be done creating simpler interfaces, less complex data structures or by empowering analysts to execute ETL jobs so that they are in greater control of their data they need in real time.

On the other side, Data users must be well trained and comfortable using data so that they are empowered to get what they need when they need it. Data training usually requires substantial effort to enhance the capabilities of the entire team, and it can take months to get everyone up to speed. This is not a quick fix - it's an investment in the future productivity of your organization.

But what about Data Person?

We are not advocating for an elimination of the Data Person role but rather a re-allocation of that persons skills. Rather than positioning him or her as reactive to the individual data needs of the organization, Data Person must be proactive in implementing easy-to-use technical solutions while training analysts to use it. Oh, and do this will also doing their regular jobs during the transition. Easy? Not so much. But give us a shout - we might be able to help.

Data persons or data users - do you have any thoughts on this? What's wrong with our approach? Are we being realistic about the capabilities of organizations to close "the waiting gap"?


  1. The worst part of it was that the software only worked intermittently and the data was not accurate. You obviously canot confront anyone about what you have discovered if the information is not right.
    data science course in malaysia
    data science certification
    data science course malaysia
    data science malaysia
    data scientist course malaysia

  2. I have bookmarked your site since this site contains significant data in it. You rock for keeping incredible stuff. I am a lot of appreciative of this site.
    pmp certification in malaysia

  3. Great post!! It's good to share this kind of articles and I hope you'll share an article about Data Science. By giving an institute like 360DigiTMG.it is one of the best institutes for certified courses.
    data science course in noida

  4. Two full endorsement for this magnificent article of yours. I've genuinely refreshing scrutinizing this article today and I figure this might be uncommon contrasted with other articles that I've examined now. On the off chance that it's not all that much difficulty prop this work up on in a comparable quality.

  5. Wonderful illustrated information. I thank you about that. No doubt it will be very useful for my future projects. Would like to see some other posts on the same subject!
    Digital Marketing Training Institutes in Hyderabad

  6. Incredibly conventional blog and articles. I am realy very happy to visit your blog. Directly I am found which I truly need. Thankful to you and keeping it together for your new post.
    data science courses in gurgaon

  7. Very informative blog post,
    Digital Marketing is the process of marketing products or services using digital technologies on the internet through computers, mobile apps ,display advertising etc. According to our research these are the companies offering best digital marketing and advertising agencies in Hyderabad.
    Best Digital Marketing companies in Hyderabad

  8. great article!! sharing these type of articles is the nice one and i hope you will share an article on data science.By giving a institute like 360DigiTMG.it is one the best institute for doing certified courses
    data science training in aurangabad

  9. Crime in London is rising five times faster than the rest of England, so it’s more important than ever to ensure your safety through hiring personal protection with us.private security

  10. This is an excellent post I seen thanks to share it. It is really what I wanted to see hope in future you will continue for sharing such a excellent post.
    data science training in malaysia

  11. Data Analytics jobs are rising in the market. Become a Data Analyst with the 360DigiTMG Data Analytics course.
    Data Analyst Course in Bangalore