May 28, 2014

The Efficient Analyst (part one): Stop moving your data around!


Many organizations collaborate by moving data and information between people, files or applications.  While movement may provide analysts with the illusion of adding value, data movement is not the same thing as data sharing. While the latter is a good thing to be nurtured, the former causes version control problems, wasted time, and analyst frustration.

In our original post on why data analysts are so inefficient, we covered the six core causes of data waste. In today's post we'll tackle the first cause: conveyance.  Below are five ways to reduce data movement and put the focus back on sharing.

1. Stop emailing (IMing, Skyping) files back and forth. Email and chat programs are some of the most popular ways to move data around. While we are all guilty of doing this every now and then, it creates huge access and version control problems for your team. Stop doing it. The most effective alternative is to utilize your company's sharing environment to store and update files. Don't have a sharing environment? Go to #4.

2. Eliminate "staging" files or sheets. A good general rule is: do not put data into a spreadsheet, database or file that does not add specific value. Many use staging files when restructuring or reformatting data before being used by someone or something else. Eliminating these types of files not only eliminates waste and confusion, but also ensures faster assembly of process documentation.

3. Be careful with "movement automation". This category can include FTP syncs, VBA macros or any other kind of automated process that pulls data from one location into another. FTP syncs often create timing bottlenecks (to be discussed in a future post) or create confusion among users. VBA macro code in Excel can cause chaos when there are structural changes in either the source or destination spreadsheets. In general, any kind of data automation must be handled with a high level of care and used only when the risk outweighs the amount of effort it takes to manually execute the task.

4. Create a sharing environment. There are many products for sharing data across organizations. Some upload files to share drives or use Sharepoint or Dropbox, while others will build custom SQL databases to store raw data. Some use Shooju. Whatever option you choose, ensuring that it is a complete solution is critical to creating any form of internal data harmony. Because of the confusion they can cause without full adoption, partial solutions can be worse than no solution at all.

5. Google it. Some organizations have no love lost for the cloud, while others are rapidly making the transition. Wherever your company is positioned, Google Drive can be an excellent solution to specific sharing problems. While seasoned data analysts may scoff at this idea, there are great ways to track changes and store raw data sets that streamline shared data flow.

A word about sharing across applications


The suggestions in this post are most effective when moving data between individuals using the same application - i.e. moving an Excel file to another Excel user. Sharing data across applications is much more difficult. In many organizations, analysts that use more advanced analytical tools like SAS often have to export results into Excel spreadsheets (creating a "staging problem") and email them (gasp!) to colleagues who are non-SAS users. Additionally, keeping this staging files updated can be a big pain.


Readers: what tips do you have to solve data sharing across applications? Leave them in the comments below.

20 comments :

  1. I am grateful for touching data breach topic. As the staff member of one of the best online data room providers I want to emphasize the problem of the private account invasion, and also I underline that it should be given more attention to that issue by the law makers.

    ReplyDelete
    Replies
    1. Big data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. IEEE Projects for CSE in Big Data But it’s not the amount of data that’s important. Final Year Project Centers in Chennai It’s what organizations do with the data that matters. Big data can be analyzed for insights that lead to better decisions and strategic business moves.

      Spring Framework has already made serious inroads as an integrated technology stack for building user-facing applications. Corporate TRaining Spring Framework the authors explore the idea of using Java in Big Data platforms.
      Specifically, Spring Framework provides various tasks are geared around preparing data for further analysis and visualization. Spring Training in Chennai

      Delete
  2. When the day dawns and you have to move your belongings from one place to another the first thing you have to do is ask yourself just how to get your stuff from your present location to your destination. Secondly, you have to find a good moving company. Movers Dallas

    ReplyDelete
  3. Overseeing of the huge information isn't less demanding then you think it takes a few methods and traps for the overseeing of huge type of unthinkable information, so in the event that you don't have the foggiest idea about these systems and trap then you ought to need to visit this website the original source Active Wizards for getting the best information researcher for you who will oversee you enormous information for you.

    ReplyDelete
  4. This blog is a punchy bit of composing, as it has a solid impact.
    Removals service

    ReplyDelete
  5. Pretty amazing work on the author's part and This blog is a five star work.
    man with van services

    ReplyDelete
  6. Imagine yesterday went to some online casino practicable online casino gambling sites and decided to ponastalgirovat. so many pleasant memories when you again see this slot machine mmm cool

    ReplyDelete
  7. Glad to be a part of your blog I have find some informative info here. Last month I Moved my home to another city and (Safe Relocation) helped me. They are the best, Highly recommended!!

    ReplyDelete
  8. I have read all the comments and suggestions posted by the visitors for this article are very fine,We will wait for your next article so only.Thanks! cheap moving companies denver

    ReplyDelete
  9. Within this webpage, you'll see the page, you need to understand this data. Moving companies near me

    ReplyDelete
  10. Extraordinary interest by female and transsexual customers and all the more as of late, because of society's mentalities, the quantity of male customers is expanding.https://www.mudanzasmaiorica.com/

    ReplyDelete
  11. In this time your only wish is to settle things as quickly as possible because right now you and your family are surrounded by a lot of mess.

    Removals Rugby

    ReplyDelete
  12. I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here! keep up the good work... Moving

    ReplyDelete
  13. As I would like to think I don't believe that Data Entry Projects themselves are tricks.input data entry

    ReplyDelete
  14. Very efficiently written information. It will be beneficial to anybody who utilizes it, including me. Keep up the good work. For sure i will check out more posts. This site seems to get a good amount of visitors. https://northamptonremovalspro.co.uk

    ReplyDelete
  15. If the employees are forced to use data that are disorganized, outdated and even misused, data entry outsource then this leads to unfortunate circumstances that even extend to the business partners, manufacturers and the customers.

    ReplyDelete
  16. Before selecting a mover, take a few minutes to think about your needs. Do you want to do most of the packing yourself, or would you prefer a full service mover who will handle all of the packing and unpacking for you? Moving services

    ReplyDelete
  17. Really a nice and informative post. Your explanation skill is really amazing. Thanks for this helpful article on moving data. Movers in Fort Worth

    ReplyDelete
  18. You should avoid hiring a moving company if any of the following situations arise Relocation services

    ReplyDelete
  19. On the surface, cardboard packaging may seem like a minor consideration for business operations - but in fact, the packaging process could be costing your company a lot of time and money if you don't have an effective strategy in place! quickboxespackaging.com

    ReplyDelete