Author Posts

June 15, 2017 at 7:34 pm

Hi guys, need your guidance in sorting the following out.

Scenario: I have a file server where new folders with files are created everyday. These folders size may range from few hundred MBs to GBs. For archiving purposes, my task is to copy yesterday's records (Folders and the files within) along with file permissions to another location. Here is my script:

# Define the variables
$today = get-date -Hour 0 -Minute 0 -Second 0 -Millisecond 0
$yesterday = get-date $today.adddays(-1) -Hour 0 -Minute 0 -Second 0 -Millisecond 0

# Getting output and Copy to another location with timestamps
get-childitem -Path D:\data\SOURCEFOLDER -recurse | where {$_.CreationTime -gt (get-date $Yesterday) -and $_.CreationTime -lt $today} | ForEach-Object {Robocopy D:\data\SOURCEFOLDER '\\servername\destination' /E /COPY:DATSOU $_.Name}

Issue: Everything works well until ForEach-Object, somehow ROBOCOPY doesn't copy the extracted content. Robocopy creates the whole folder structure from the SOURCE directory. I tried COPY-ITEM, but it has some limitations, so had to use ROBOCOPY. Any ideas what I am doing wrong here?

June 15, 2017 at 11:54 pm

I don't think it is powershell. It's the /e switch from robocopy

June 16, 2017 at 5:13 am

Hi Frank, I have tried /S switch too, it hasn't worked either. What i can see, robocopy is going through each and every folder (thousands) but not picking up the one delivered by powershell.

June 16, 2017 at 7:08 am

Wrap the robocopy command into Invoke-Command and then run through each object.

ForEach-Object {Invoke-Command -Command "Robocopy D:\data\SOURCEFOLDER '\\servername\destination' /E /COPY:DATSOU $_.Name"}

Thank you.

June 16, 2017 at 8:03 am

Thanks Kiran, will try it on Monday and let you know.

June 16, 2017 at 3:39 pm

don't use the /s switch either... you code copied just the files for me with just removing the /e switch

June 17, 2017 at 4:12 am

Thanks Frank. I think the U (auditing) switch in COPY /DATSOU could be at fault too. Anyways, will try and let you know.

June 19, 2017 at 12:37 am

Hi Kiran,
It fails with the following error:

Invoke-Command : Cannot bind parameter 'ScriptBlock'. Cannot convert the "Robocopy D:\data\SOURCEFOLDER 'C:\temp\testcopy' /E /COPY:DATSOU FOLDER1.Name" value of type "System.String" to type
"System.Management.Automation.ScriptBlock".
At line:4 char:176
+ ... mmand -Command "Robocopy D:\data\SOURCEFOLDER 'C:\temp\testcopy' /E /COPY:DATSOU ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Invoke-Command], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.InvokeCommandCommand

June 19, 2017 at 5:48 am

Sorry, use Invoke-Expression, it's not Invoke-Command

June 19, 2017 at 7:51 am

It creates the entire folder structure from SOURCE at the destination, and then goes through each and every file in the subfolders which i don't want. Imagine going through hundreds and thousands of folders looking for content. What i want Robocopy to do is just pick the results produced by powershell and not to look into every folder.

June 19, 2017 at 7:56 am

Tried this too, no luck. I did a test run on files, it works great, but when folders and subfolders get involved, it doesn't do the job.
The interface and logs says it is copying files, but the destination has got nothing. Checked the disk space to see if some change is taking place somewhere else, but no. Any other suggestions?

June 19, 2017 at 8:54 am

Then remove -recurse flag from from Get-ChildItem in your script.

June 19, 2017 at 10:10 am

why you don't use robocopy's /MAXAGE /MINAGE switches and play with get-childitem ?
you can generate values for these switches with PS and let ROBOCOPY filter files.

and, for better parameter handling try to launch robocopy this way

$parameters = @($source, $destination)
if (...) { $parameters += '/MaxAge:{0}' -f $age }
# [...]
& robocopy $parameters

June 19, 2017 at 7:05 pm

Hi Max, can you please elaborate it bit further, sorry I don't have scripting background. What is IF (...) statement doing? Thanks.

June 19, 2017 at 8:25 pm

"If" intend the non-binding of adding some parameters to robocopy 🙂
if you will be satisfied not creation but modification time, you need something like "robocopy source destination /minage:1 /maxage:2" without powershell at all
or look at other examples https://social.technet.microsoft.com/wiki/contents/articles/1073.robocopy-and-a-few-examples.aspx

the main message: be best way to call robocopy: use "& robocopy $parameterarray" syntax

because robocopy doesn't support creationtime nor file-with-path selection,
for exactly your variant I can suggest this:

$source= 'd:\1'
$destination= 'd:\2'
$today = (Get-Date).Date
$yesterday = $today.AddDays(-1)
get-childitem $source -Recurse | where-object { $_.CreationTime -gt $yesterday -and $_.CreationTime -lt $today } | Foreach-Object { robocopy (Split-Path $_.FullName) $destination $_.Name }

It copies file WITHOUT folder structure into destination
for maintaining folder structure there is need some magic for destination folder tree creation

June 19, 2017 at 8:47 pm

and the magic is:

get-childitem $source -Recurse |
where-object {
    $_.CreationTime -gt $yesterday -and $_.CreationTime -lt $today
} | Foreach-Object {
    $s = (Split-Path $_.FullName)
    $d = $destination + $s.Substring($source.Length)
    robocopy $s $d $_.Name
}

the additional switches for robocopy at you taste

June 20, 2017 at 8:42 am

I've been away from the office today, will give it a go tomorrow and let you know the outcome. Hopefully, it works!

June 22, 2017 at 3:39 am

Hi Max,

The 2nd script worked, but i think we are missing something from it. It took 3 hrs 20 mins to copy just 4.56 GB of data from 1 SATA volume to another. Normal copy & paste takes only approx 2 mins.
I started the transcript to get some logs and found the following:

Host Application: C:\Windows\system32\WindowsPowerShell\v1.0\PowerShell_ISE.exe
Process ID: 3420
**********************
Transcript started, output file is C:\temp\logs.txt
Transcript started, output file is C:\temp\logs.txt
Transcript started, output file is C:\temp\logs.txt
Transcript started, output file is C:\temp\logs.txt
——————————————————————————-
——————————————————————————-
——————————————————————————-
——————————————————————————-
ROBOCOPY :: Robust File Copy for Windows
ROBOCOPY :: Robust File Copy for Windows
ROBOCOPY :: Robust File Copy for Windows
ROBOCOPY :: Robust File Copy for Windows
——————————————————————————-
——————————————————————————-
——————————————————————————-
——————————————————————————-
Started : Thursday, 22 June 2017 9:54:20 AM
Started : Thursday, 22 June 2017 9:54:20 AM
Started : Thursday, 22 June 2017 9:54:20 AM
Started : Thursday, 22 June 2017 9:54:20 AM
Source : D:\data\SOURCEFOLDER\
Source : D:\data\SOURCEFOLDER\
Source : D:\data\SOURCEFOLDER\
Source : D:\data\SOURCEFOLDER\
Dest : C:\temp\testcopy\
Dest : C:\temp\testcopy\
Dest : C:\temp\testcopy\
Dest : C:\temp\testcopy\
Every line in log file is x4. Does that mean, it is copying multiple (x4) times? I thought it could be the recurse parameter, however after removing the parameter the script would not copy any folder. Thoughts?

June 22, 2017 at 10:11 am

Don't know why it repeat many times but this code really call robocopy one time per one file, this is the bottleneck.
Seems you start-transcrpt somewhere inside cycle but nut before script run

So, as I'm already said, you should see if you can use Modification time or step away from robocopy.

what problems you have with copy-item ?
If it is lfn sopport look on PSAlphaFS module

June 23, 2017 at 6:54 am

I started transcript at the beginning of the script. The problem i had with Copy-item was, it wouldn't copy the timestamps; unless you have any suggestions.

June 26, 2017 at 4:16 am

Any more thoughts?