Help with Start-job

This topic contains 13 replies, has 2 voices, and was last updated by Profile photo of Olaf Soyk Olaf Soyk 2 months ago.

  • Author
    Posts
  • #62186
    Profile photo of Goldy
    Goldy
    Participant

    So im trying to do a robocopy script that uses start job. But before I delved into actually doing the script I figured what's a better way to test it, with get-childitem.

    Please assume $ulog is a log file and location, so example c:\_logs\userid.log

    And assume admin_path is a unc path

    \\server\location\homedrive

    
    $scriptblock = "get-childitem -path $admin_path -recurse | out-file $($ulog)" 
    $scriptblock
    
        Start-Job -scriptblock { $scriptblock | out-null }
    

    Also, this is part of a loop with 50 lines in it.

    What am I missing that I'm not getting any log file.

  • #62190
    Profile photo of Olaf Soyk
    Olaf Soyk
    Participant

    You have to pass your log file as an argument to your scriptblock

    https://blogs.msdn.microsoft.com/powershell/2009/12/29/how-to-pass-arguments-for-remote-commands/

    • #62238
      Profile photo of Goldy
      Goldy
      Participant

      Well, that seesm to be true, but I can also use the using: scope. I managed to get it work if I put the $using: scope int the actualy start-job line, but if I try to pass a variable into it with $using:scriptblock – that doesn't work.

      So you are saying I need an argument?

  • #62242
    Profile photo of Goldy
    Goldy
    Participant

    Actually the easiest way to do this is finally using the $using scope along with invoke-expression

    $scriptblock = do blah

    start-job {invoke-expression $using:scriptblock}

    • #62244
      Profile photo of Olaf Soyk
      Olaf Soyk
      Participant

      Personally I would prefer something like this:

      Start-Job -ScriptBlock {get-childitem -path $args[0] -recurse | out-file $args[1] } -ArgumentList $admin_path,$ulog

      IMHO that's more readable.

  • #62277
    Profile photo of Goldy
    Goldy
    Participant

    If I'm going to be switching this to robocopy, i was hoping I can also capture $lastexitcode

    I tried to build a string in here-string and then run invoke-expression inside start-job. No luck.

    What would be the best way to capture the lastexitcode with an if statement. Iw as trying to prevent myself from looping through all the logs to review for errors.

    
    $user = "userid"
    
    $logs = "C:\temp\_logs"
    $elog = $logs + "\$($user)_error.log"
    
    $scriptblock = @"
    robocopy c:\fake h:\powershell *.* /e /r: /w:1
    
    # Check exit code
    if (($LASTEXITCODE -gt 1) -and ($LASTEXITCODE -lt 16))
    {
        $RoboCopyMessage = "EXITCODE: 1-15, Warning, Please refer to the copy log"
    	ac -path $elog -value $robocopymessage
    }
    if ($LASTEXITCODE -eq 16)
    {
        $RoboCopyMessage = "EXITCODE: 16, Error, Please refer to the copy Log"
    	ac -path $elog -value $robocopymessage
    }
    "@
    
    Start-job {invoke-expression $using:scriptblock} 
    
    
  • #62284
    Profile photo of Olaf Soyk
    Olaf Soyk
    Participant

    What's wrong with the straight forward approach? You will have to catch the job afterwards anyway.

    Start-job {robocopy $args[0] $args[1] *.* /e /r:1 /w:1 /  /LOG:$args[2] } -ArgumentList $Source, $Destination, $Logfile
    • #62287
      Profile photo of Goldy
      Goldy
      Participant

      ^^
      I actually am leaning more towards your approach instead of using invoke-expression.

      The problem is I'm doing a copy of 300+ shares to new locations and instead of writing a loop to go thru the robocopy log I was hoping to easily put this in my start-job statement where if there is an error it will tell me by dumping exit code to an error file.

      I have it working in small form, but on larger scale, I need it I think.

      So lets go with your route

      Start-job {robocopy $args[0] $args[1] *.* /e /r:1 /w:1 /  /LOG:$args[2]; if ($lastexitcode -eq 16 {ac -path $args[3] -value $lastexitcode } -ArgumentList $Source, $Destination,$errorpath
      

      That works, but I was hoping to open that up to all of robocopys exit codes

  • #62350
    Profile photo of Olaf Soyk
    Olaf Soyk
    Participant

    This probably should have been my first question to you: Why would you like to use a job (or jobs) for running a file/folder copy task? Mostly jobs are used to speed up things we usually have to wait for. In my experiences file copy tasks actually never lack on computing power. Usually the I/O capacity of the storage system is the limiting bottle neck. I suspect that even if you manage to run several copy jobs at the same time it will not reduce the overall time consumed by these tasks in summary.

    So instead of putting a lot of effort to run these copy jobs in parallel you could just run them one after another – maybe as a scheduled task or just in another Powershell console you minimized to the taskbar.

    BTW: parsing the output of a robocopy log is much easier than you might think. 😉

  • #62352
    Profile photo of Goldy
    Goldy
    Participant

    Olaf –

    I'm not doubting that parsing just single threaded robocopy logs aren't easy, but I don't just have one source, I could have 6 different sources and 12 different destinations so in this case, the bottleneck is not the storage, but the actual server i'm running this from.

    I thought about single threading it, but mgmt wants this to happen faster. But i'm struggling with the error log parsing.

    I can then loop all 300 ids through a check log script and for each copy, I have one log, so it's id = id.log We can easily parse that but I was looking to gather the $lastexitcode and make my life somewhat easier...

  • #62379
    Profile photo of Olaf Soyk
    Olaf Soyk
    Participant

    But i'm struggling with the error log parsing.

    If you need help with that ... 😉 😀

    • #62391
      Profile photo of Goldy
      Goldy
      Participant

      Did i miss something?

      I think you would agree with me that error log on single thread is easier. Its how to concatenate it with regards to catching lastexitcode and writing only a portion of that.

    • #62475
      Profile photo of Olaf Soyk
      Olaf Soyk
      Participant

      Did i miss something?

      Probably not – maybe I did. Sorry, english is not my first language.

      I think you would agree with me that error log on single thread is easier. Its how to concatenate it with regards to catching lastexitcode and writing only a portion of that.

      I used to think that exit codes from robocopy others than 0 or 1 are actually not helpfull at all. I have to invetigate the log anyway.

  • #62424
    Profile photo of Goldy
    Goldy
    Participant

    So as of now, i need to have an error log check loop for each file and i have some code that will pull out the errors.

    I was hoping to do this as part of start-job but i'm now separating this out

You must be logged in to reply to this topic.