How to add fault tolerance?

This topic contains 6 replies, has 4 voices, and was last updated by  Aapeli Hietikko 8 months, 3 weeks ago.

  • Author
    Posts
  • #62842

    Michael Clayton
    Participant

    Good morning!

    I've compiled some scripts that I've had some help with to stop a service, make sure it's stopped before starting the service back up, I think I have that part working. What I would like to add is if the service never comes back up after 3 try's at 5 min a piece (300 sec)that it will give up and email out that the service restart has failed so we can manually intervene.

    Any help would greatly be appreciated. I'm a novice at best.

    $ServiceName = 'Adobe Acrobat Update Service'
    $arrService = Get-Service -Name $ServiceName
    while ($arrService.Status -eq 'Running')
    {
    Stop-Service -Name $ServiceName
    write-host $arrService.Status
    write-host 'Service Stopping'
    start-sleep -seconds 10
    $arrService.Refresh()
    if ($arrService.Status -ne 'Running')
    {
    Write-Host 'Service is now Stopped'
    }
    }
    while ($arrService.Status -ne 'Running')
    {

    Start-Service $ServiceName
    write-host $arrService.status
    write-host 'Service starting'
    Start-Sleep -seconds 10
    $arrService.Refresh()
    if ($arrService.Status -eq 'Running')
    {
    Write-Host 'Service is now Running'
    }

    }
    $arrService.Refresh()
    $Subject = ($arrService.Status)

    $body = Restart-MyService | Out-String

    $email = @{
    From = "@email.com"
    To = "@email.com"
    Subject = $Subject
    SMTPServer = "smtp.******.com"
    Body = $body
    }

    send-mailmessage @email

  • #62859

    Don Jones
    Keymaster

    It'd help immensely if you could format your code, as indicated in the bullet points above the posting text box. It's difficult to follow the logic as-is.

  • #62869

    Michael Clayton
    Participant
    $ServiceName = 'Adobe Acrobat Update Service'
    $arrService = Get-Service -Name $ServiceName
    while ($arrService.Status -eq 'Running')
    {
    Stop-Service -Name $ServiceName
    write-host $arrService.Status
    write-host 'Service Stopping'
    start-sleep -seconds 10
    $arrService.Refresh()
    if ($arrService.Status -ne 'Running')
    {
            Write-Host 'Service is now Stopped'
        }
        }
    while ($arrService.Status -ne 'Running')
    {
    
        Start-Service $ServiceName
        write-host $arrService.status
        write-host 'Service starting'
        Start-Sleep -seconds 10
        $arrService.Refresh()
        if ($arrService.Status -eq 'Running')
        {
            Write-Host 'Service is now Running'
        }
    
    }
    $arrService.Refresh()
    $Subject = ($arrService.Status) 
    
    $body = Restart-MyService | Out-String
    
    $email = @{
    From = "emai.com"
    To = "emai.com"
    Subject =  $Subject
    SMTPServer = "smtp.*******.com"
    Body = $body
    }
    
    send-mailmessage @email

    Sorry about that, I should of read.

  • #62877

    Aapeli Hietikko
    Participant

    I'm not 100% sure if I understood the goal but could it go like this

    $ServiceName = 'Adobe Acrobat Update Service'
    $i=0
    
    do
    {
        Stop-Service -Name $ServiceName
        sleep 10
        $status = (Get-Service -Name $ServiceName).Status
    
    } while ($Status -eq 'Running')
    
    Write-output 'Service is now Stopped'
    
    
    do
    {
        start-Service -Name $ServiceName
        sleep 300
        $i++
        $status = (Get-Service -Name $ServiceName).Status
    
    } while (
            $status -eq 'Stopped' -or $i -ge 3
            )
    
    if ($status -eq 'Running') {
                        Write-output 'Service is now Running'
                        }
    
    Else {
        
        $body = @"
    Hola,
    
    Your service restart script failed to start $ServiceName
    
    Please check the status
    
    Br
    PS Automation
    "@
    
    
        $email = @{
            From = "emai.com"
            To = "emai.com"
            Subject =  "[SuperFailure] $ServiceName did not start"
            SMTPServer = "smtp.*******.com"
            Body = $body
            }
        
        send-mailmessage @email
    }
    
  • #62878

    Michael Clayton
    Participant

    Thank you very much Aapeli!

    It's very close to what I want from what I can read, I'll test it in a moment. The only other thing I Would need is for it to also email upon success.

    • #62884

      Aapeli Hietikko
      Participant
      $ServiceName = 'Adobe Acrobat Update Service'
      $i=0
      
      do
      {
          Stop-Service -Name $ServiceName
          sleep 10
          $status = (Get-Service -Name $ServiceName).Status
      
      } while ($Status -eq 'Running')
      
      
      do
      {
          start-Service -Name $ServiceName
          sleep 300
          $i++
          $status = (Get-Service -Name $ServiceName).Status
      
      } while (
              $status -eq 'Stopped' -or $i -ge 3
              )
      
      if ($status -eq 'running') {
              $output = "$ServiceName is now Running"
              }
        else  {
              $output = "$ServiceName did not start"
              }
      
      
          
      $body = @"
      Hola,
      
      $output
      
      Br
      PS Automation
      "@
      
      
      $email = @{
          From = "emai.com"
          To = "emai.com"
          Subject =  "[SuperFailure] $ServiceName did not start"
          SMTPServer = "smtp.*******.com"
          Body = $body
          }
          
      send-mailmessage @email
      
      
  • #62880

    Rob Simmers
    Participant

    Another example:

    $ServiceName = 'Foxit Reader Service'
    $service = Get-Service -Name $ServiceName
    $timeout = 0
    
    while (($service.Status -ne 'Running') -and ($timeout -le 60)) {
        $service.Refresh()
        start-sleep -seconds 10
        "{0} is {1}. Paused for {2} of 60 seconds." -f $ServiceName, $service.Status, $timeout
        $timeout += 10
    }
    
    if ( $service.Status -eq 'Running' ) {
        "{0} service is up and running and good to go" -f $ServiceName
    }
    else {
        "{0} service didn't start in the time specified" -f $ServiceName
    }
    

You must be logged in to reply to this topic.