Welcome › Forums › General PowerShell Q&A › merge Powershell CustomObjects
- This topic has 17 replies, 2 voices, and was last updated 1 month, 2 weeks ago by
Participant.
-
AuthorPosts
-
-
December 1, 2020 at 8:22 am #275586
Hi, I have a question, maybe someone has an idea.
I’m reading two JSON files and would like to merge the information into one PSCustomObject but only the single values should be changed if the information already exists or a new propertiy shoud be added if the property doesent exist.
$Json1 is a global information and the values of $Json2 have the last word if there are equal keys or if additional keys musst be added.
So the information from $Json2 is leading, but should only overwrite or enrich information.
Example of JSONs.
$Json1:
{
“Hostname”: “Server1”,
“Port”: “”5001”,
“Paths”:{
“Path1”: “C:\\test\123”,
“Path2”: “C:\\test\456”
“Path3”: “C:\\test\456”
“Pfad4”: “C:\\test\456”
}
}
$Json2:
{
“Hostname”: “Server2”,
“Port”: “”5001”,
“Paths”:{
“Path1”: “E:\\Prod\123”,
“Path2”: “E:\\Prod\456”
}
}
I read the two files and convert it using ConvertFrom-Json
So the information is now in the variables $Json1 and $Json2
I now merge the two variables into one object.
Code:
$Object = [ordered] @{}
foreach ($Property in $Json1.PSObject.Properties) {
$Object += @{$Property.Name = $Property.Value}
}
foreach ($Property in $JSON2.PSObject.Properties) {
try{
$Object += @{$Property.Name = $Property.Value}
}catch{
$Object.$($Property.Name) = $Property.Value
}
}
}The result of the newly created object should look as follows:
“Hostname”: “Server2”
“Port”: “”5001”,
“Paths”:@{
“Path1”: “E:\\Prod\123”
“Path2”: “E:\\Prod\456”
“Path3”: “C:\\test\456”
“Path4”: “C:\\test\456”}
But what comes out is:
“Hostname”: “Server2”
“Port”: “”5001”,
“Paths”:@{
“Path1”: “E:\\Prod\123”
“Path2”: “E:\\Prod\456”
}
I dont understand how to modfy only the Path1 and Path2 but keeping Path3 and 4 from the JSON1
The hostname from the JSON2 was taken over and the port remained unchanged as desired.
Only the values of path1 and path2 were changed with the JSON2 information, but path3 and path4 were deleted.
The code above deletes exactly this information path3 and path4. Everything else is replaced correctly.
So I have problems to adjust only the nested values. Everything on the first level works, but as soon as another level is added, Path-> Path1 the information of Path is completely replaced.
This is quite clear in the code, but I do not really have the idea how to solve the problem and only change the information of path1 and 2 but not deleting path3 and Path4 from the $json1
Result should look like this.
The result of the newly created object should look like this:
“Hostname”: “Server2”
“Port”: “”5001”,
“Paths”:@{
“Path1”: “E:\\Prod\123”
“Path2”: “E:\\Prod\456”
“Path3”: “C:\\test\456”
“Path4”: “C:\\test\456”}
I could access the path information with the name $Object.Paths.Path1, but the script should be generic because the use case is just to merge different Json files into one hash table/PSObject and if the same property is present in both Json files, the desired Json file information (json2) overwrites the values of the other one, regardless of the level depth. (Paths->;Path1…)
for ideas I would be thankful
best regards
Rolf
-
This topic was modified 1 month, 2 weeks ago by
WotansHammer29.
-
This topic was modified 1 month, 2 weeks ago by
WotansHammer29.
-
This topic was modified 1 month, 2 weeks ago by
WotansHammer29.
-
This topic was modified 1 month, 2 weeks ago by
WotansHammer29.
-
This topic was modified 1 month, 2 weeks ago by
-
December 1, 2020 at 12:28 pm #275667
Merge isn’t really the right term. Merging is typically taking two objects, say for instance the same HostName, and if you had memory in one object and cpu in another object and you want CPU and Memory associated with that hostname. Enough for semantics, it appears you want to overwrite the paths from the other record making them authoritative. Here is an example:
PowerShell12345678910111213141516171819202122232425262728293031323334$Json1 = @"{"Hostname": "Server1","Port": "5001","Paths":{"Path1": "C:\\test\\123","Path2": "C:\\test\\456","Path3": "C:\\test\\456","Path4": "C:\\test\\456"}}"@ | ConvertFrom-Json$Json2 = @"{"Hostname": "Server2","Port": "5001","Paths":{"Path1": "E:\\Prod\\123","Path2": "E:\\Prod\\456"}}"@ | ConvertFrom-Jsonforeach ($path in $Json1.Paths.PSObject.Properties){$newVal = $Json2.Paths.PSObject.Properties | Where-Object {$_.Name -eq $path.Name}if ($newVal) {'Setting value {0} to {1} for property {2}' -f $path.Value, $newVal.Value, $path.Name$path.Value = $newVal.Value}}$Json1Output:
PowerShell1234567PS C:\Users\rasim> c:\Users\rasim\Desktop\temp.ps1Setting value C:\test\123 to E:\Prod\123 for property Path1Setting value C:\test\456 to E:\Prod\456 for property Path2Hostname Port Paths-------- ---- -----Server1 5001 @{Path1=E:\Prod\123; Path2=E:\Prod\456; Path3=C:\test\456; Path4=C:\test\456} -
December 1, 2020 at 1:13 pm #275700
Hi, thank you for your answere…..
I merge two json files into one object. Both files provide information and in case of collisions the second file (Json2) should win and modify the key or value or key/value pair.
The first part of the script builds an object with key value pairs. Because a new object is created, there are no collisions adding new keys.(dubilcate keys not possible)The second function should trys to add all keys and values from the json2 and replace them with new values from the json2 or add new keys from the json2 if the new key from json2 are not present.
If a key is allready present the catch part will just modify the value of this key ($Object.$($Property.Name) = $Property.Value)
This works well with one level json files. (hostname, port…)My json files have 2 levels and i haven’t always the information of the json structure…I just know that the json have a maximum deep of 2 levels…
Level1
“Hostname”: “Server2”,
“Port”: “5001”,
“Paths”:{Level 2
“Path1”: “E:\\Prod\\123”,
“Path2”: “E:\\Prod\\456”so ..I have the problem that I have to replace the Keys/values from the second level (Paths -> Path1, Path2, Path3, …) without the names of the properties or keys ($Json2.Paths.PSObject.Properties )
The output of your solution for the Path path is that what i need….The Hostname should also changed
Thanks a lot
Rolf
-
December 1, 2020 at 2:44 pm #275736
Ok, is ugly but is a frist try….mybe the right way…
PowerShell123456789101112131415161718192021$Object = [ordered] @{}foreach ($Property in $Json1.PSObject.Properties) {$Object += @{$Property.Name = $Property.Value}}foreach ($Property in $Json2.PSObject.Properties) {try{$Object += @{$Property.Name = $Property.Value}}catch{$PropName = $($Property.Name)if($Property.TypeNameOfValue -notmatch 'System.String'){foreach ($path in $Object.$PropName.PSObject.Properties){$newVal = $Json2.$PropName.PSObject.Properties | Where-Object {$_.Name -eq $path.Name}if ($newVal) {'Setting value {0} to {1} for property {2}' -f $path.Value, $newVal.Value, $path.Name$path.Value = $newVal.Value}}}else{$Object.$($Property.Name) = $Property.Value}}} -
December 1, 2020 at 3:12 pm #275739
This a very unorthodox way of doing things to overwrite everything about an object with another object. This should be a recursive function if you wanted to handle any depth, but this should work:
PowerShell123456789101112131415161718192021222324252627282930313233343536373839404142434445464748$Json1 = @"{"Hostname": "Server1","Port": "5001","Paths":{"Path1": "C:\\test\\123","Path2": "C:\\test\\456","Path3": "C:\\test\\456","Path4": "C:\\test\\456"}}"@ | ConvertFrom-Json$Json2 = @"{"Hostname": "Server2","Port": "5001","Paths":{"Path1": "E:\\Prod\\123","Path2": "E:\\Prod\\456"}}"@ | ConvertFrom-Jsonforeach ($prop in $Json1.PSObject.Properties){if ($prop.Value -is [PSCustomObject]) {foreach ($subProp in $prop.Value.PSObject.Properties) {$newSubVal = $Json2."$($prop.Name)".PSObject.Properties | Where-Object {$_.Name -eq $subProp.Name}if ($newSubVal) {'Setting value {0} to {1} for sub property {2}' -f $subProp.Value, $newSubVal.Value, $subProp.Name$subProp.Value = $newSubVal.Value}}}else {$newVal = $Json2.PSObject.Properties | Where-Object {$_.Name -eq $prop.Name}if ($newVal) {'Setting value {0} to {1} for property {2}' -f $prop.Value, $newVal.Value, $prop.Name$prop.Value = $newVal.Value}}}$Json1Output:
PowerShell123456789PS C:\Users\rasim> c:\Users\rasim\Desktop\temp.ps1Setting value Server1 to Server2 for property HostnameSetting value 5001 to 5001 for property PortSetting value C:\test\123 to E:\Prod\123 for sub property Path1Setting value C:\test\456 to E:\Prod\456 for sub property Path2Hostname Port Paths-------- ---- -----Server2 5001 @{Path1=E:\Prod\123; Path2=E:\Prod\456; Path3=C:\test\456; Path4=C:\test\456} -
December 2, 2020 at 3:44 am #275892
Hi, the script does not replace everything. It replace only things are present in both json files with the information of the json2 or it adds additional key/value pairs from the json2 if not present in the json1.
The use case is that the JSON1 is a kind of “Global” Parameter file and the Json2 a kind of local parameter file. Thats means that all scripts use the global json (lets call it json1) for default values like default paths, defaul server, default databases and so on. The local json (json2) have usually only additional values but for some scripts we must override some parameters deliverd from the json1 because there use maybe different paths or server or databases and so on. We talk about a of a handfull parameters that are overwritten.
In the most cases, the jsons are different and have only additional informations.
many thanks for yours support – it was very helpfull
If you have a better idea to handle the use case….let me know
Best regards
Rolf
-
December 2, 2020 at 3:51 am #275895
you don’t like my try{} Catch{} 😀
-
December 2, 2020 at 5:44 am #275916
A better example of the jsons….my first description was a bit too general or too simple
PowerShell123456789101112131415161718192021222324252627282930313233343536$Json1 = @"{"Hostname": "Server1","FileServer": "Server2","Repository": "Server3","DBServer": "Server4","Port": "5001","SQLDriver": "{Some String}","LogPaths": "E:\\SomeFileLocation\\","MuchMoreProperties": "MuchMoreEntries","....": "...", #means additonal properies not present in the json2"Paths":{"Path1": "C:\\test\\123","Path2": "C:\\test\\456","Path3": "C:\\test\\456","Path4": "C:\\test\\456"}}"@ | ConvertFrom-Json$Json2 = @"{"Hostname": "Server2","Port": "5002","SQLServer": "Serverx","FileExtension": ".dll","Filename": "SomeName","SubPath": "\\SomePath\\","AndSoOn": "SomeValues","....": "...", #means additonal properies not present in the json1"Paths":{"Path1": "E:\\Prod\\123","Path2": "E:\\Prod\\456"}}"@ | ConvertFrom-JsoBest regards
rolf
-
This reply was modified 1 month, 2 weeks ago by
WotansHammer29.
-
This reply was modified 1 month, 2 weeks ago by
-
December 2, 2020 at 11:32 am #276003
Taking two objects properties and combining them into one object with properties from object 2 values being authoritative over object 1 property values.
PowerShell12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182$Json1 = @"{"Hostname": "Server1","FileServer": "Server2","Repository": "Server3","DBServer": "Server4","Port": "5001","SQLDriver": "{Some String}","LogPaths": "E:\\SomeFileLocation\\","MuchMoreProperties": "MuchMoreEntries","NotInJson2": "SomeOtherValue","Paths":{"Path1": "C:\\test\\123","Path2": "C:\\test\\456","Path3": "C:\\test\\456","Path4": "C:\\test\\456"}}"@ | ConvertFrom-Json$Json2 = @"{"Hostname": "Server2","Port": "5002","SQLServer": "Serverx","FileExtension": ".dll","Filename": "SomeName","SubPath": "\\SomePath\\","AndSoOn": "SomeValues","NotInJson1": "SomeOtherValue","Paths":{"Path1": "E:\\Prod\\123","Path2": "E:\\Prod\\456"}}"@ | ConvertFrom-Json#Empty hashtable for new object$newObjProps = @{}#Find matching properties if they exist in Json1 and overwrite with Json2 values#or add properties that only existing in Json1foreach ($prop in $Json1.PSObject.Properties){if ($prop.Value -is [PSCustomObject]) {$newObjSubProps = @{}foreach ($subProp in $prop.Value.PSObject.Properties) {$newSubVal = $Json2."$($prop.Name)".PSObject.Properties | Where-Object {$_.Name -eq $subProp.Name}if ($newSubVal) {'Overwriting value {0} to {1} for sub property {2}' -f $subProp.Value, $newSubVal.Value, $subProp.Name$newObjSubProps.Add($subProp.Name, $newSubVal.Value)}else {'Adding value {0} for sub property {1}' -f $subProp.Value, $subProp.Name$newObjSubProps.Add($subProp.Name, $newSubVal.Value)}}'Adding property {0}' -f $prop.Name$newObjProps.Add($prop.Name, (New-Object -TypeName PSObject -Property $newObjSubProps))}else {$newVal = $Json2.PSObject.Properties | Where-Object {$_.Name -eq $prop.Name}if ($newVal) {'Overwriting value {0} to {1} for property {2}' -f $prop.Value, $newVal.Value, $prop.Name$newObjProps.Add($prop.Name, $newVal.Value)}else {'Adding value {0} for property {1}' -f $prop.Value, $prop.Name$newObjProps.Add($prop.Name, $prop.Value)}}}#Identify properties in Json2 that are not in the hash table for the new objectforeach ($prop in $Json2.PSObject.Properties | Where-Object -FilterScript {$newObjProps.Keys -notcontains $_.Name}){$newObjProps.Add($prop.Name, $prop.Value)}$myNewObject = New-Object -TypeName PSObject -Property $newObjProps$myNewObjectOutput:
PowerShell1234567891011121314151617181920212223242526272829303132PS C:\Users\rasim> c:\Users\rasim\Desktop\temp.ps1Overwriting value Server1 to Server2 for property HostnameAdding value Server2 for property FileServerAdding value Server3 for property RepositoryAdding value Server4 for property DBServerOverwriting value 5001 to 5002 for property PortAdding value {Some String} for property SQLDriverAdding value E:\SomeFileLocation\ for property LogPathsAdding value MuchMoreEntries for property MuchMorePropertiesAdding value SomeOtherValue for property NotInJson2Overwriting value C:\test\123 to E:\Prod\123 for sub property Path1Overwriting value C:\test\456 to E:\Prod\456 for sub property Path2Adding value C:\test\456 for sub property Path3Adding value C:\test\456 for sub property Path4Adding property PathsNotInJson2 : SomeOtherValueMuchMoreProperties : MuchMoreEntriesDBServer : Server4Repository : Server3Port : 5002AndSoOn : SomeValuesSQLDriver : {Some String}FileServer : Server2Hostname : Server2FileExtension : .dllFilename : SomeNameSubPath : \SomePath\LogPaths : E:\SomeFileLocation\SQLServer : ServerxPaths : @{Path3=; Path1=E:\Prod\123; Path4=; Path2=E:\Prod\456}NotInJson1 : SomeOtherValue-
This reply was modified 1 month, 2 weeks ago by
Rob Simmers.
-
This reply was modified 1 month, 2 weeks ago by
-
December 3, 2020 at 6:41 am #276147
Hi, nice but the output is not correct.
Paths : @{Path3=; Path1=E:\Prod\123; Path4=; Path2=E:\Prod\456}
Path 3/4 are empty
-
December 3, 2020 at 6:47 am #276153PowerShell12345678910111213141516171819202122232425262728293031function Combine-Objects {[CmdletBinding()]param ([Parameter(Mandatory=$true)][Object]$Json1,[Parameter(Mandatory=$true)][Object]$Json2)$Object = [ordered] @{}foreach ($Property in $Json1.PSObject.Properties) {$Object += @{$Property.Name = $Property.Value}}foreach ($Property in $Json2.PSObject.Properties) {try{$Object += @{$Property.Name = $Property.Value}}catch{$PropName = $($Property.Name)if($Property.TypeNameOfValue -notmatch 'System.String'){foreach ($Val in $Object.$PropName.PSObject.Properties){$newVal = $Json2.$PropName.PSObject.Properties | Where-Object {$_.Name -eq $Val.Name}if ($newVal) {$Val.Value = $newVal.Value}}}else{$Object.$($Property.Name) = $Property.Value}}}return [pscustomobject] $Object}
-
December 3, 2020 at 10:33 am #276279
The verbose output is correct, so line 55 should reference the $subProp, not $newSubVal, this…
PowerShell1$newObjSubProps.Add($subProp.Name, $newSubVal.Value)should be:
PowerShell1$newObjSubProps.Add($subProp.Name, $subProp.Value) -
December 4, 2020 at 3:25 am #276477
Thanks for you support and… a nice solution.
But the code i posted do the same….Try it, same result 🙂
PowerShell123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172function Merge-Objects {[CmdletBinding()]param ([Parameter(Mandatory=$true)][Object]$Json1,[Parameter(Mandatory=$true)][Object]$Json2)$Object = [ordered] @{}foreach ($Property in $Json1.PSObject.Properties) {$Object += @{$Property.Name = $Property.Value}}foreach ($Property in $Json2.PSObject.Properties) {try{$Object += @{$Property.Name = $Property.Value}}catch{$PropName = $($Property.Name)if($Property.TypeNameOfValue -notmatch 'System.String'){foreach ($Val in $Object.$PropName.PSObject.Properties){$newVal = $Json2.$PropName.PSObject.Properties | Where-Object {$_.Name -eq $Val.Name}if ($newVal) {$Val.Value = $newVal.Value}}}else{$Object.$($Property.Name) = $Property.Value}}}return [pscustomobject] $Object}$Json1 = @"{"Hostname": "Server1","FileServer": "Server2","Repository": "Server3","DBServer": "Server4","Port": "5001","SQLDriver": "{Some String}","LogPaths": "E:\\SomeFileLocation\\","MuchMoreProperties": "MuchMoreEntries","NotInJson2": "SomeOtherValue","Paths":{"Path1": "C:\\test\\123","Path2": "C:\\test\\456","Path3": "C:\\test\\456","Path4": "C:\\test\\456"}}"@ | ConvertFrom-Json$Json2 = @"{"Hostname": "Server2","Port": "5002","SQLServer": "Serverx","FileExtension": ".dll","Filename": "SomeName","SubPath": "\\SomePath\\","AndSoOn": "SomeValues","NotInJson1": "SomeOtherValue","Paths":{"Path1": "E:\\Prod\\123","Path2": "E:\\Prod\\456"}}"@ | ConvertFrom-JsonMerge-Objects -Json1 $Json1 -Json2 $Json2rolf
-
This reply was modified 1 month, 2 weeks ago by
WotansHammer29.
-
This reply was modified 1 month, 2 weeks ago by
-
December 4, 2020 at 10:34 am #276573
Glad your code is working, a couple of tips:
- While you could argue everything is an object in Powershell, $object would normally be a PSObject. That is created with the type accelerator at the end [pscustomobject]. You’re working with a hashtable or dictionary, key\value pairs. Using += is typically a performance hit and not recommended, especially when the hashtable has methods to manage (add\remove\set) the hashtable.
PowerShell12345678910111213141516171819202122PS C:\Users\rasim> $hash = [ordered] @{}PS C:\Users\rasim> $hash.Add('Prop1','value1')PS C:\Users\rasim> $hash['Prop2'] = 'Value2'PS C:\Users\rasim> $hash['Prop2'] = 'New Value'PS C:\Users\rasim> $hash.Set_Item('Prop1','Another New Value')PS C:\Users\rasim> $hashName Value---- -----Prop1 Another New ValueProp2 New ValuePS C:\Users\rasim> $hash.Contains('Prop2')TruePS C:\Users\rasim>
- try\catch here is a bit dangerous as anything could error and you’re just executing another code block. Recommend that you use .Contains (for ordered hash) or .ContainsKey for a standard hash to see if a key exists
- Name functions singular, not plural. Get-Process, Get-Service, etc.
Here is a modified version:
PowerShell1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677function Merge-Object {[CmdletBinding()]param ([Parameter(Mandatory=$true)][object]$Json1,[Parameter(Mandatory=$true)][object]$Json2)$hash = [ordered] @{}foreach ($Property in $Json1.PSObject.Properties) {$hash[$Property.Name] = $Property.Value}foreach ($Property in $Json2.PSObject.Properties) {if ($hash.Contains($Property.Name)) {$hash[$Property.Name] = $Property.Value}else {$PropName = $($Property.Name)if ($Property.TypeNameOfValue -notmatch 'System.String'){foreach ($Val in $hash.$PropName.PSObject.Properties){$newVal = $Json2.$PropName.PSObject.Properties | Where-hash {$_.Name -eq $Val.Name}if ($newVal) {$Val.Value = $newVal.Value}}}else{$hash[$Property.Name] = $Property.Value}}}[pscustomobject]$hash}$Json1 = @"{"Hostname": "Server1","FileServer": "Server2","Repository": "Server3","DBServer": "Server4","Port": "5001","SQLDriver": "{Some String}","LogPaths": "E:\\SomeFileLocation\\","MuchMoreProperties": "MuchMoreEntries","NotInJson2": "SomeOtherValue","Paths":{"Path1": "C:\\test\\123","Path2": "C:\\test\\456","Path3": "C:\\test\\456","Path4": "C:\\test\\456"}}"@ | ConvertFrom-Json$Json2 = @"{"Hostname": "Server2","Port": "5002","SQLServer": "Serverx","FileExtension": ".dll","Filename": "SomeName","SubPath": "\\SomePath\\","AndSoOn": "SomeValues","NotInJson1": "SomeOtherValue","Paths":{"Path1": "E:\\Prod\\123","Path2": "E:\\Prod\\456"}}"@ | ConvertFrom-JsonMerge-Object -Json1 $Json1 -Json2 $Json2 - While you could argue everything is an object in Powershell, $object would normally be a PSObject. That is created with the type accelerator at the end [pscustomobject]. You’re working with a hashtable or dictionary, key\value pairs. Using += is typically a performance hit and not recommended, especially when the hashtable has methods to manage (add\remove\set) the hashtable.
-
December 4, 2020 at 12:47 pm #276618
Thanks for the Tips…I will keep it in mind.
But….The modified code delivers wrong output 🙂
@{Path1=E:\Prod\123; Path2=E:\Prod\456}
-
December 4, 2020 at 1:01 pm #276636
Find and replace can be a pain…should be where-object…
PowerShell1| Where-hash {$_.Name -eq $Val.Name} -
December 4, 2020 at 1:50 pm #276639
🙂 Sometimes….i know.
But that wasn’t the problem 🙂
Same result 🙂
The first Foreach overwrites the Path result:
PowerShell1234foreach ($Property in $Json2.PSObject.Properties) {if ($hash.Contains($Property.Name)) {$hash[$Property.Name] = $Property.Value}-
This reply was modified 1 month, 2 weeks ago by
WotansHammer29.
-
This reply was modified 1 month, 2 weeks ago by
-
-
AuthorPosts
- You must be logged in to reply to this topic.