Given Infinite stream (incoming stream) where size is not know, we need to find the k random samples where probability of randomness of each element is same.
Solution:
This is class of problem called online algorithm. Online algorithm deals with the infinite streams (infinite means size of data set is not known). This class is famous where source is continuous getting data.
Some popular hits in this category are:
1. Find the running median in stream of integers.
2. Find top N searches in search engines.
Let us see why can't we solve this problem in a trivial way. If i want to solve this problem in trivial way i need to just get K random number in a set of N numbers. Now for the above problem we do not know the size of the data so we can't get the K random numbers (or medians in case of median problems).
Let us try different approach with example. Suppose i have to select 2 numbers and i have stream up to 2 numbers, then answer will contain the two numbers. Now the moment i get the third data then we have two options either to escape that 3rd number or to replace the either of 2 numbers with 3rd number. Now while replacing we need to find a way which number to replace. So whole problem lies in finding the probability with which new number is candidate for set and then the probability of replacing the numbers in the list (list of k set). This is called reservoir sampling. Let us go ahead and try to find an optimal way to find the probability.
First i will put algorithm then i will show the correctness of algorithm (source : Wikipedia
array R[k]; // result
integer i, j;
// fill the reservoir array
for each i in 1 to k do
R[i] := S[i]
done;
// replace elements with gradually decreasing probability
for each i in k+1 to length(S) do
j := random(1, i); // important: inclusive range
if j <= k then
R[j] := S[i]
fi
done
#include<iostream>
#include<ctime>
using namespace std;
#define SIZE 10
#define STREAM_AVERAGE 10
int reservoirSample(int sample, int* samples, int size, int count)
{
if(count < size)
samples[count] = sample;
else
if((rand()%count) < size)
samples[rand()%size] = sample;
return ++count;
}
int main()
{
int count = 0;
int samples[SIZE];
int sample;
int i = 0;
srand(time(NULL));
cout << "Sample Stream: " << endl;
while(
(count < SIZE) ||
(rand()%STREAM_AVERAGE > 0)
)
{
sample = rand()%1000;
cout << sample << " ";
count = reservoirSample(sample, samples, SIZE, count);
}
cout << endl;
cout << "Total samples: " << count << endl;
cout << "Output samples: " << endl;
for(i = 0;i < SIZE;i++)
cout << samples[i] << " ";
cout << endl;
}
Proof of correctness: Now when sample k + i arrives then probability of occurrence in one of the slot will be 1/ (k+i). since there are k outout slots. So probability of chances in being output is k / (k + i).
chances of not in output = 1 - k / (k + i) = i / (k + i). so logically it makes sense if we say generate random no between 1 to k + i if it less than k then it is in output otherwise not.
Proof by induction.
Assuming the prior stage worked correctly each element in the output should have K/(K+i-1) chance of being present.
There is i/(K+i) chance that prior output will be unchanged this round. Hence each element has Ki/(K+i)(K+i-1) chance of being output if the unchanging choice is made for this stage.
There is K/(K+i) chance that this something in the output will change
And for each element there is (K-1)/K chance it will not be replaced thus a total of (K-1)/K * K/(K+i) * K/(K+i-1) chance it will not be removed.
So conversely there is a 1/K * K/(K+i) * K/(K+i-1) chance that it will be replaced.
Hence the net chance of the existing elements making it to the next round is the sum of the chance that nothing changed with the chance that it was not the one that changed.
= Ki/(K+i)(K+i-1) + (K-1)K/(K+i-1)(K+i)
=> (Ki + (K-1)K)/(K+i-1)(K+i)
=> K(i+K-1)/(K+i-1)(K+i)
=> K/(K+i)