Tag Archives: Screwups

Debate over NSA collecting information … can the media begin to report substantively?!!

7 Jun

So this business of the NSA collecting data should come as no surprise to anyone. The media is having a field day! The issue is whether or not the intel community is doing this legally. Has the FISA court done anything illegal? The court is guided by a set of rules that are mean to be transparent, known to the non-intel world, and approved by Congress. Did they follow these rules when allowing the NSA to collect what it collected? Did they restrict the use of that data appropriately? These are the questions — remember after 9-11 when everyone was asking (with outrage) why we had not connected the dots, but back to my area of concern…

CNN has pulled out their favorite privacy pundit – Jim Harper from the CATO Institute. Jim is well spoken, and very learned in the field of privacy and policy. However, he makes a statement in this interview that I find incredible – he says that collecting all the data from every American’s phone calls “can’t possibly be useful for link-based investigation.” Really, I cannot think of a better way of using phone call data than in linked based analysis. Methinks you need to stick with policy Jim!! Anyone out there care to explain this comment?

As a matter of policy, there are probably some questions to be answered.  The FISA courts have been criticized for approving everything without question. I would like the news agencies to focus on that, and whether or not the court is working as envisioned to protect our privacy.

Have a look at this post that is homeland security oriented – they are harvesting things differently here, but… same privacy concerns.

Advertisement

Tags: , , , ,

This should come as no suprise… Using Excel for complex analysis on an ongoing basis is asking for trouble!

22 Apr

This report on how using Excel has caused some major miscalculations should come as no surprise… Excel exists because it is pervasive, easy to use and can be applied to a range of decision making activities. However, have you ever had the experience of trying to create a repeatable, defensible and transparent report using excel WITHOUT having to make sure you had done it correctly? The attached article talks about a number of mistakes. I have had a number of discussions over the years with companies that are struggling with whether or not to implement a BI system, and if so to what extent should it provide structure and guidance to the process of using Excel?

The easy implementation of BI is to implement a tool such as Tableau that in essence takes spreadsheets and allows you to pivot the data and visualize more easily that one could in excel. I realize that Tableau does more than that now, but that is how it started and most people appear to use it that way still. This gives you great looking dashboards, and allows you to roll around in the data to bubble up insights. However, it does nothing to address the quality of the report and the issues raised by the article.

At the other end of the spectrum are enterprise level tools that do a great job of locking down the source data, and tracking everything that happens at the desktop to make the final report.These tools are focused on creating the SAME report with exactly the same inputs and calculations as all previous times. To the extent changes are made, they are tracked, and capabilities exist to flag and route changes for review and approval. The downside of course is that they often limit what the user can do with the data.

Somewhere in the middle is the happy spot. To the extent tools are not able to support the requirements for transparency, traceability, and defensibility, these requirements must be addressed through policy, process and standards.  Most of the enterprise tools are configurable to create a well defined set of gates between which analysts and report creators can have complete flexibility.

In the cases mentioned in this article, the technology exists to create the safeguards required. However, the user communities were able to resist change, and management – for whatever reason – did not make the decision to invest in underlying data management, BI and analytical capabilities. In a data driven world, it is only a matter of time before that comes back to bite you.

Tags: , , ,

%d bloggers like this: