Download all specific files and create structure for sites and subsites in a SC

This topic contains 1 reply, has 2 voices, and was last updated by  postanote 2 weeks, 1 day ago.

  • Author
    Posts
  • #97870

    john adams
    Participant

    Hi Guy's,

    I have found a script that is useful for what i am trying to achieve but i have been unable to amend it to get the desired result. The current script only iterates through the top site and libraries within and creates the corresponding folders and downloads the relevant files to these folders. However, what i would like to do is instead of the top site, i would like to iterate through all sites and subsites and libraries and download and create the structure accordingly. So far when i have tried i am able to create the folders for the sites but this puts them at the top level so end up with a url 1000's of characters long, not good. So am trying achieve that if there are 7 sites under the top level then they are created under the TL folder accordingly and if each site has a subsite then they are placed into their relevant site folder and library within etc..
    I'm still learning PS so at the mo i'm butchering big hefty scripts to get the results i need. Below is the current script i have made slight tweaks but reverted to the working copy that downloads just the top level site and libraries.
    Granted this script looks at the site, but even when i try and change to use the sutecollection allwebs it's still not working.
    Any help greatly appreciated. Thanks
    # This script will extract all of the documents and their versions from a site. It will also
    # download all of the list data and document library metadata as a CSV file.

    Add-PSSnapin Microsoft.SharePoint.PowerShell -erroraction SilentlyContinue
    #
    # $destination: Where the files will be downloaded to
    # $webUrl: The URL of the website containing the document library for download
    # $listUrl: The URL of the document library to download

    #Where to Download the files to. Sub-folders will be created for the documents and lists, respectively.
    $destination = "D:\Export"

    #The site to extract from. Make sure there is no trailing slash.
    $site = "http://yoursitecollection/yoursite"

    # Function: HTTPDownloadFile
    # Description: Downloads a file using webclient
    # Variables
    # $ServerFileLocation: Where the source file is located on the web
    # $DownloadPath: The destination to download to

    function HTTPDownloadFile($ServerFileLocation, $DownloadPath)
    {
    $webclient = New-Object System.Net.WebClient
    $webClient.UseDefaultCredentials = $true
    $webclient.DownloadFile($ServerFileLocation,$DownloadPath)
    }

    function DownloadMetadata($sourceweb, $metadatadestination)
    {
    Write-Host "Creating Lists and Metadata"
    $sourceSPweb = Get-SPWeb -Identity $sourceweb
    $metadataFolder = $destination+"\"+$sourceSPweb.Title+" Lists and Metadata"
    $createMetaDataFolder = New-Item $metadataFolder -type directory
    $metadatadestination = $metadataFolder

    foreach($list in $sourceSPweb.Lists)
    {
    Write-Host "Exporting List MetaData: " $list.Title
    $ListItems = $list.Items
    $Listlocation = $metadatadestination+"\"+$list.Title+".csv"
    $ListItems | Select * | Export-Csv $Listlocation -Force
    }
    }

    # Function: GetFileVersions
    # Description: Downloads all versions of every file in a document library
    # Variables
    # $WebURL: The URL of the website that contains the document library
    # $DocLibURL: The location of the document Library in the site
    # $DownloadLocation: The path to download the files to

    function GetFileVersions($file)
    {
    foreach($version in $file.Versions)
    {
    #Add version label to file in format: [Filename]_v[version#].[extension]
    $filesplit = $file.Name.split(".")
    $fullname = $filesplit[0]
    $fileext = $filesplit[1]
    $FullFileName = $fullname+"_v"+$version.VersionLabel+"."+$fileext

    #Can't create an SPFile object from historical versions, but CAN download via HTTP
    #Create the full File URL using the Website URL and version's URL
    $fileURL = $webUrl+"/"+$version.Url

    #Full Download path including filename
    $DownloadPath = $destinationfolder+"\"+$FullFileName

    #Download the file from the version's URL, download to the $DownloadPath location
    HTTPDownloadFile "$fileURL" "$DownloadPath"
    }
    }

    # Function: DownloadDocLib
    # Description: Downloads a document library's files; called GetGileVersions to download versions.
    # Variables
    # $folderUrl: The Document Library to Download
    # $DownloadPath: The destination to download to
    function DownloadDocLib($folderUrl)
    {
    $folder = $web.GetFolder($folderUrl)
    foreach ($file in $folder.Files | ?{ $_.Item["Colour"] -eq 'Red' })
    {
    #Ensure destination directory
    $destinationfolder = $destination + "\" + $folder.Url
    if (!(Test-Path -path $destinationfolder))
    {
    $dest = New-Item $destinationfolder -type directory
    }

    #Download file
    $binary = $file.OpenBinary()
    $stream = New-Object System.IO.FileStream($destinationfolder + "\" + $file.Name), Create
    $writer = New-Object System.IO.BinaryWriter($stream)
    $writer.write($binary)
    $writer.Close()

    #Download file versions. If you don't need versions, comment the line below.
    GetFileVersions $file
    }
    }

    # Function: DownloadSite
    # Description: Calls DownloadDocLib recursiveley to download all document libraries in a site.
    # Variables
    # $webUrl: The URL of the site to download all document libraries
    function DownloadSite($webUrl)
    {
    $web = Get-SPWeb -Identity $webUrl

    #Create a folder using the site's name
    $siteFolder = $destination + "\" +$web.Title+" Documents"
    $createSiteFolder = New-Item $siteFolder -type directory
    $destination = $siteFolder

    foreach($list in $web.Lists)
    {
    if($list.BaseType -eq "DocumentLibrary")
    {
    Write-Host "Downloading Document Library: " $list.Title
    $listUrl = $web.Url +"/"+ $list.RootFolder.Url
    #Download root files
    DownloadDocLib $list.RootFolder.Url
    #Download files in folders
    foreach ($folder in $list.Folders)
    {
    DownloadDocLib $folder.Url
    }
    }
    }
    }

    #Download Site Documents + Versions
    DownloadSite "$site"

    #Download Site Lists and Document Library Metadata
    DownloadMetadata $site $destination

  • #97876

    postanote
    Participant

    Have a look at the IIS WebAdministration module...

    'docs.microsoft.com/en-us/powershell/module/webadministration/?view=win10-ps'

    ... to list all the sites and their site UNC path(s), or other info, that you'd put into a collection and feed to your script.

    Something like this should get you started on such a use case. Using that Path property to create your directory structure.

    Get-Website | select name,id,state,physicalpath, 
    @{n="Bindings"; e= { ($_.bindings | select -expa collection) -join ';' }} ,
    @{n="LogFile";e={ $_.logfile | select -expa directory}}, 
    @{n="attributes"; e={($_.attributes | % { $_.name + "=" + $_.value }) -join ';' }}
    
    name         : TestSite
    id           : 2
    state        : Started
    physicalPath : C:\inetpub\TestSite
    Bindings     : http 192....:80:TestSite.contoso.com
    LogFile      : %SystemDrive%\inetpub\logs\LogFiles
    attributes   : name=TestSite;id=2;serverAutoStart=True;state=1
    

You must be logged in to reply to this topic.