Author Posts

January 1, 2012 at 12:00 am

by karmista at 2013-01-05 06:01:49

Curious Case of Windows Forms Authentication via PowerShell –
How to login to a webpage via powershell and download a million pdf documents?

Hi Powershell Gurus,

Need your assistance with this interesting problem in this regard.
We need to login to a webpage via powershell and download nearly a million pdf documents. There are 2 levels of authentication – Negotiate (via a standard pop-up & Forms (via the UN/Pwd field in the login.aspx)

Last week, I had a post on this wonderful forum & got a brilliant idea to pursue a programmatic approach.

The web site is hosted on IIS 6. It has "Integrated Windows Authentication" check box checked (IIS 6).

web.config Authentication & Authorization info looks like this –

"




"

Below Power Shell code authenticates & gets to the login.aspx page. However, I am unable to perform the "Forms" authentication from the login.aspx page.

If I attempt to directly download URL2 i.e. $webclient.DownloadFile($url2,$path), I get a HTTP 500 Internal Server Error message.

Need your assistance in getting through these authentication via tweaks to the below code.

$url = "http://10.20.30.40/myApp/login.aspx"
$path = "D:\temp\myfile1.pdf"

#$url2 = "http://10.20.30.40/myApp/servpdf.aspx?docno=1234567&allToTiff=true"

$webclient = new-object System.Net.WebClient
$credCache = new-object System.Net.CredentialCache

$creds = new-object System.Net.NetworkCredential("username","password", "domain")
$credCache.Add($url, "Negotiate", $creds)
$webclient.Credentials = $credCache

$webclient.DownloadFile($url,$path)
#$webclient.DownloadFile($url2,$path)

by DonJ at 2013-01-08 08:23:57

This is going to be pretty complicated, and you're getting pretty deep into .NET programming. For a PowerShell forum, I don't think you're necessarily going to get a lot of answers. You might consider asking at StackOverflow.com, which has a developer following and folks who've probably done this.

Forms authentication doesn't accept a credential in the way you're providing. With Forms auth, you're not responding to an HTTP authentication prompt, you're entering a username and password into form fields and pressing a Submit button. The Web server authenticates that on the back-end, and then issues you one or more cookies in the response to show you've been authenticated. You have to store those cookies and re-present them in every new request. A Web browser does all of this for you; given that you're essentially writing you're own Web browser, it's on you.

I can't help but wonder if some Download Manager-style browser plugin might not be a better way for you to do this. I've seen tools that can crawl a site and download files in bulk. Otherwise, you're going to be doing quite a lot of coding to make this work.