Author Posts

August 8, 2017 at 3:54 pm

Alright, I have csv document. Each row of the csv has a timestamp, along with various performance metrics as they were when recorded at that time.

What I want to be able to do is consume the csv and perform a logic test on it of this kind of nature:

If said metric is above a specified threshold for a specified amount of time = True

I can easily setup a logic statement comparing two numbers/metrics, but I wasn't sure how to add in the component as to whether that condition is true for a certain amount of time.

I could use a bit of direction and would greatly appreciate any help anyone can provide.

Thanks in advance

August 8, 2017 at 3:55 pm

You mean, you're not sure how to calculate the amount of time taken from the timestamp? I'm not following what you want – maybe a concrete example would help?

August 8, 2017 at 4:11 pm

For example

Csv might look something like this:

time,KB/sec,IOPS,Avg. Latency ms

8/6/2017 10:30,10616.79606,117.2243058,35.63250298

8/6/2017 10:20,11337.82872,149.4084982,28.55670254

8/6/2017 10:17,12276.47016,172.2729097,25.43457276

8/6/2017 10:14,10125.01863,106.2432985,37.38492431

8/6/2017 10:11,12185.1857,127.452136,32.40784617

So I might want to know if the latency is consistently over a certain amount for a couple of hours for instance. Stated another way, if there is a duration of time(based on the timestamps) that the latency exceeded a certain amount. I am hoping there might be some logic involving datetime I don't know, and not having to setup a multilayer logic system with potentially checking each value multiple times with loops.

I hope that makes more sense.

August 8, 2017 at 4:16 pm

Yeah "consistently" is the problem, right? Because you can't just write a comparison. This is going to be just as complex and you may have feared. You're going to have to first establish what you think "too long" is. But then you're going to have to loop through every line and do some tracking.

It might help to write down your logic in English form.

For the current line, is the latency over threshold? If not, reset all counters. If so, increment a counter. If the counter ever gets over some threshold, display an alert.

So that logic would let you track whether or not latency exceeded "x" for more than "y" lines in a row.

If you want the logic to be more complex than that... I'd start by writing it down like that. But there's not some simple command or anything you're going to be able to use. You're going to have to code up the logic, although I don't see multiple loops.

August 8, 2017 at 7:35 pm

Fortunately I only have to find if it occurs at all and not every time for each time period so I can "break" after I find one such occurance. It didn't turn out as bad as I thought it might...

$sorted = $file |Sort-Object time
$lastentry=$sorted |Select-Object -Last 1
$outputtable=@()
$outputtext=@()

#Average write latency
$highwritelatency= $sorted | Where-Object {$_."avg. write latency ms" -gt 20}
$groupname=$lastentry.object
ForEach($filterwrite in $highwritelatency)
{
$starttime=$null
$starttime=$filterwrite.time
$endtime=$starttime.addhours(2)

$timeframe=$sorted | Where-Object {($_.time -gt $starttime) -and ($_.time -lt $endtime)}

$test=$timeframe | Where-Object {$_."avg. write latency ms" -lt 20}

IF(-not $test)
{
$outputtext+="$groupname `t Write Latency > 20%"}
break
}

Thanks for the help