In this article, we will discuss the different techniques that you can use to predict when a computer hardware system becomes saturated with data. These are not the only predictive analytics methods available for such problems but they offer some of the most useful and efficient ones. As it is an important consideration in many systems, it is worth understanding how to tell when saturation occurs so that you can take any necessary corrective actions before your system crashes! One method is to model the system in question as being composed of a finite number of bins or buffers and then take measurements when you fill these with an aggregate amount. The power-law distribution is often used here, which assumes that data can be stored at different rates within any given time frame:
Fill each successive bin with a proportionally smaller share of data
Measure the time until saturation is reached
If all bins are filled, then the system has become saturated and you need to take action. Binary search can be used as a predictive analytics technique in this case:
Start by filling bin 0 (assume it’s at capacity) with an aggregate amount of data; measure how long before overflow occurs for that bin; refill it with half the overall allocated allotment and measure again. Repeat these steps while continually reducing your allotted allowance by 50% each iteration until either 100 iterations have been made or overflow detection is found! With binary search, we see that if there were only two levels of measure available, overflow detection would be found in the first iteration
To avoid this, look for ways to break down your data into smaller chunks
If you can’t do that, then use predictive analytics technologies such as moving average or weighted histograms
By quantifying how much space each bin is currently taking up and predicting where it will end up given historic averages of past performance history, we can identify when saturation might occur
Bin size: 12 GB (allocated) Average fill rate per day: 500 MB/day = 50 days before overflow occurs. Predictive Analytics Technique: Moving Averages. The system won’t become saturated until about 51 days have passed with no action taken!
Bin Size of 16GB allocated PAtent Analytic Techniques-Weighted Histograms. The system will not become saturated for another 41 days even if there is no change in the current pattern or activity level, and then only if the current level of data is maintained
Citations: – there are sources that support all three techniques. For example, a “moving average” technique from Johanson and Richardson (2002) can be found in this paper on page 16; weighted histograms from Wang et al. (2004); and least-recently-used algorithm for managing memory by Chen et al., 2006).
The Predictive Analytics Techniques: – Moving Averages, weighted histograms, and the least-recently-used algorithm for managing memory. The system won’t become saturated until about 51 days have passed with no action taken! Bin Size 12GB allocated Patent Analytic Techniques-Weighted Histograms. The system will not become saturated for another 41 days even if there is no change in the current pattern.
Moving Average: a predictive analytic technique that calculates an average of several numbers by adding them together and dividing by how many numbers there are… If we were to use this process on our example data set, it would be calculated as follows: (57+54+52)/(12) = 46 so far –
Weighted Histogram: a predictive analytic technique that plots the number of times an event occurs at various threshold values… For example, if we want to vary between 0 and 100 bytes on our system then we would take the total byte size as follows (0-100) so far. The value for this is 64/16 = .375 or 37.50% of available memory has been used up yet!
Memory Least Recently Used Algorithm: a process that determines which blocks are likely candidates for deletion by analyzing those with outbound pointers in other parts of the computer’s memory space and not being accessed over time–that means it last read what you wrote there 41 days ago!–It will delete all the blocks with outbound pointers and then keep an eye on what is written there.
