The Goldilocks Data Zone
Over the past couple of years there has been much argument from proponents of both ‘big’ and ‘small’ data to justify the merits of their relative positions and solutions. Pulse Mining believes it has found a ‘sweet spot’ between these camps in what might be termed the ‘Goldilocks Data Zone’.
Big Data is… the large volume of data, both structured and unstructured, that inundates a business on a day-to-day basis. “It is being generated by everything around us at all times. Every digital process and social media exchange produces it. Systems, sensors and mobile devices transmit it. Big data is arriving from multiple sources at an alarming velocity, volume and variety.” (IBM)
Small Data is… “small datasets that are capable of impacting decisions in the present.” (Dataversity)
The Goldilocks Data Zone… according to Ash Bosworth, Director of Analytics and Innovation at Pulse Mining Systems, “is more about getting the right data than the size of the data pool from which it comes.”
Pulse Mining has impeccable credentials in terms of working with business data. The company has developed and refined its own mining industry specific ERP (Enterprise Resource Planning) solution over a period of 30 years, has more than 20 years experience in providing analytics and visualisations of this data to help facilitate effective business decision making, and in 2016 was internationally recognised as one of the ‘25 Most Promising Business Intelligence Solution Providers’ by APAC CIO Outlook magazine.
The transactional and machine data quantities that Pulse Mining’s ERP and analytics solutions process and manage are enormous, for instance:
- Transactional data managed by the ERP is produced by approximately 2,500 sub-programs all integrated to handle the complete business data needs of mining companies from pit to ship, across multiple business entities, and incorporating the full gamut of data and management needs for vital areas such as production, plant maintenance, payroll, HSEC (Health, Safety, Environment and Community), marketing and logistics.
- One example of IoT (Internet of Things)/machine data linkage to Pulse Analytics is from the Continuous Miners used in Gate Road development for underground coal mining. Data is gathered from over 300 sensors on each machine, with each sensor capturing and transmitting data every second… that’s a lot of data!
In view of this experience and expertise Pulse Mining is well-placed to comment on business data trends, tools and techniques.
So… what is the Goldilocks Data Zone?
The short answer is that this is the data that is pulled or pushed from Big Data sources (ERP’s, IoT, Apps, machines, databases, data warehouses and stores, the cloud, etc.), filtered by asking the right questions and formatted to act as Small Data, in that it is capable of impacting decisions here and now.
How does Pulse Mining leverage it?
Pulse Mining uses a 2-tier analytics engine with inbuilt governance mechanisms to automatically refine and validate structured and unstructured data based on pre-defined parameters and models allowing real-time access of data for assorted requirements, such as Enterprise Reporting, Predictive Analytics, Interactive Dashboards, Visual Discovery, Mobile Analytics to cater for a variety of user experiences and needs. This also allows new views or outputs to be developed to harness the power of third-party applications such as spreadsheets or rich visualisation tools such as Tableau or Cognos.
How do you get the right data?
Ultimately it boils down to asking the right questions. This means having both a broad and deep knowledge of the industry, empathy for its pain-points, open and ongoing dialogue with clients, being a good listener and being a creative solution provider.