Try something like this:
Assumptions
(1) Bad data points are not at the start or end of the data set
(2) Data is sampled at a constant rate
See the graph for a comparison of the before and after data. Remember to put the xls file and my getnextdoor function is the same dirrectory as the file below when you run it.
Regards Elarto
Code:
clc;clear;
fname = 'try.xls';
sheetName='Sheet1';
xls = xlsread(fname, sheetName);
Ref = xls(:,2);%get second row
BC = xls(:,3);%get thrid row
%calc abs of difference
ABS_Refj_Refi = abs(Ref(2:end)-Ref(1:end-1));
%95th percentile
p = prctile(ABS_Refj_Refi,85);
bad_flag = find(ABS_Refj_Refi > p);%position of bad data
bad_index_list = getNextDoor(bad_flag);%see my custom function
%we get all indexes that are bad and how many consective indexes there are.
out = BC;%save new variable
for i = 1:length(bad_index_list(:,1))% for each section of bad data
bad_index = bad_index_list(i,1);%bad index list
num_consec = bad_index_list(i,2);%how many times they repeat
L = out(bad_index-1);%left value
R = out(bad_index+num_consec+1);%right value
interp_points = interp1([0;1],[L;R],linspace(0,1,num_consec+3));%interp points
new_points = interp_points(2:end-1)';%take relevant points
out(bad_index:bad_index+num_consec) = new_points;%put new points back into saved points
end
disp('Processed data points')
disp(out)
plot(BC,'b')
hold on
plot(out,'rx')
hold off
legend('old','new')