Avoiding Partial Configurations - ideas?

This topic contains 4 replies, has 3 voices, and was last updated by  Sergei 3 months ago.

  • Author
    Posts
  • #79339

    Sergei
    Participant

    As mentioned in DSC book, partial configurations should be avoided for several reasons. I tried to avoid them when setting up our DSC enviroment however I have a nagging feeling that it may not be scalable enough.

    Our DSC Enviroment uses PullServers to distribute the mofs. This is how it is set up at the moment:

    – LCM Mofs are generated when new hosts are added to configurationData. For ConfigurationNames parameter, hostname is used as below:

            ConfigurationRepositoryWeb PullServer {
                    ServerURL                  = $configurationData.services.DSCPullServer.DSCServerURL
                    RegistrationKey            = $configurationData.services.DSCPullServer.DSCRegistrationKey
                    AllowUnsecureConnection    = $false 
                    ConfigurationNames         = @("$($node.NodeName)")
                }
    

    – There is one master Configuration file. This file uses include files for other Configurations as below:

    configuration mainConfiguration {
        param(
            [Parameter(Mandatory = $false)]
            [ValidateNotNullOrEmpty()]
            [PSCredential]$credential,
        )
    
        Node ($AllNodes | Where-Object {$_.memberOfServices -contains "WebServer"}).NodeName
        {
          . $(Join-path $ConfigurationPath WebServer.ps1)
        }  
    
        Node ($AllNodes | Where-Object {$_.memberOfServices -contains "SQL"}).NodeName
        {
          . $(Join-path $ConfigurationPath SQL.ps1)
        }  
    
    }
    

    When this configuration is running, it generates one MOF file per host.
    Having one configuration file and one configurationName per host is the only way I could avoid using partial configurations and still have scalable solution. However now I have to rerun entire configuration even if I want to apply a small change.
    I would like to find out what other people have done to avoid the partial configuration but have a more flexible and modular approach?

  • #79343

    Don Jones
    Keymaster

    What you're running into is not a scalability problem with your approach per se, it's a problem with how you, as the person making it all happen, can or cannot scale. This is a _tooling_ problem, meaning you've found a solution that works, but you're still implementing it manually, so it's a PITA.

    Rerunning the entire thing – even for a minor change – shouldn't be a problem if you've got the right tools (scripts, maybe) to automate it for you. That's the next place to invest – making some tooling to streamline the process.

    Alternately, look into Tug, the open-source DSC pull server. This isn't zero effort, but it would enable you to build yourself a smarter pull server that – as one example of what you COULD code up – dynamically create MOFs on the fly, look up configuration artifacts in a database (combining them on the server, on-demand – so, basically "partials, only better"), etc.

    • #79390

      Sergei
      Participant

      Thank you Don

      We have a release pipeline in place and entire DSC process is pretty much automated already.When someone checks in the code, the build agents build the configuration, run tests, generate mofs files and distribute them to load balanced pull servers.
      Interesting point about Tug – I will definetly investigate it.

  • #79423

    Geraldo
    Participant

    Isn't the solution just add parameter to your script, and manipulating the "AllHosts". If you don't want to change all hosts, pass only the hosts you want to change, but if your change..... I might be missing something here.

  • #79501

    Sergei
    Participant

    Thank you Geraldo

    You are absolutely right. This is the most obvious way to workaround it and you pointed me in a right direction:)

    We have a ConfigurationData data stored in files as below:

    ├───AllNodes
    │       client1.psd1
    │       client2.psd1
    │
    ├───Certificates
    │   ├───Client
    │   │       client1.domain.local.cer
    │   │       client2.domain.local.cer
    │   │
    │   └───Server
    │       ├───pullserver1
    │       │       dsc.domain.local.cer│       │
    │       └───pullserver2
    │               dsc.domain.local.cer
    │
    ├───Credentials
    │       elevateduser.psd1
    │ 
    ├───Services
    │       CA.psd1
    │       DSCPullServer.psd1
    │
    └───SiteData
            site1.psd1
            site2.psd1
    

    Release pipeline process will work like this:
    – User edits main Configuration file ( shown in the first comment)
    – User checks out files to source control
    – Automated process builds ConfigurationData datastructure using the information from the folder structure above.
    – Automated process runs Configuration using the Configuration Data datastructure.

    Now I need to think how can we can amend this workflow to be able to change content of AllNodes dynamically.

You must be logged in to reply to this topic.