Windows

How to Automate FTP Uploads with PSFTP

Posted on by Adam Fowler

Many vendors and companies still transfer data via FTP. It could be transactional data, user data, or a myriad of other things. Hopefully they’re using SFTP or FTPS (which are different ways of achieving secure FTP transmissions) rather than FTP, for similar data transfer security reasons on HTTP vs HTTPS.

A common use case I come across, is user management. Uploading basic user data like names, email addresses and employee numbers so a vendor can update records in their cloud based product for your staff to use. If you’re using a cloud service and don’t have user automation sorted – then ask them how you can achieve it – it’s much more enjoyable to set up automation, than do repeating mindless tasks.

Assuming you have details on what to send and where, you’ll need to work out how to automate FTP uploads. Note that this is a much less secure method – you’re saving the password in plain text. Alternatives do exist such as what’s demonstrated in this YouTube video below using a Public Key and Private Key Pair, but require the ability to create a .ssh folder on the FTP server. I’d rather do it this way:

Getting the other end to do what you want isn’t always possible in the real world, so you need to consider the risks if you need to save a password in plain text somewhere (saved in a Scheduled Task as you’ll see below). They’re obtainable if someone can get onto the server (or a backup of the server, or connect to Task Scheduler remotely), which should be heavily locked down anyway, and the password for this might be saved in a password database those same staff members have access to anyway.

If those credentials were obtained by another party, what could they do? If the FTP site cleans up the data instantly that’s uploaded, then they could potentially upload whatever they wanted. Look at a worst case and decide if you’re comfortable with having the account credentials saved this way, or need to find another approach.

Again, consider these risks, try to implement the most secure method you can, and raise any concerns with management/your boss. Assuming this is a scenario where you can’t do it more securely:

Instructions

First, you’ll need software. I use PSFTP – part of PuTTY, a free and open source solution. Download the full installer, as there’s a few components of PuTTY we need.

Next, you’ll need the login details of the FTP site:
Host: e.g. sftp.contoso.com
Username
Password

Open a Command Prompt, navigate to the location that contains sftp.exe and type:

sftp username@host -pw password

You’ll probably first be prompted with a message saying ‘The server’s host key is not cached in the registry’ with some details on the fingerprint. If you’re sure you’re connecting to the right server, you can say ‘y’ to ‘Store key in cache’. Once saved, you won’t be prompted for this on the same computer/user.

At this stage, we’re just making sure you can sign in and get past the key stage. If this works, you’ll now need to work on a batch file to pass through all the commands you want to do.

In this example, I’ll be going into a folder and uploading a file. Open notepad and type your commands, which you can first test in your active connection:

cd inbound
mput filename.csv
quit

Pretty simple stuff. Save your notepad file (we’ll call it batchfile.txt), and if you haven’t already disconnected from your SFTP session, do so with the ‘quit’ command.

Connect back to the SFTP site, but this time we’ll specify the batch file to rin after connecting:

sftp username@host -pw password -b batchfile.txt -batch

I’ve also added -batch on the end to specify it’s an automated batch job – this will cause SFTP to exit on a prompt, rather than be running forever waiting for an input. You can try without -batch first if you want to test and see the prompts, but you’ll need to run this command manually rather than triggering from a Scheduled Task.

If this works as expected, great! You can automate the SFTP task – the final step is to schedule it to run, which I usually use the native Scheduled Tasks in Windows to do.

If your scheduled task is running under a different account than what you tested with, then you’ll need to do that initial host key saving – easiest way is to launch Command Prompt as that user, and connect to the FTP site.

Default Printer At First Logon via Group Policy

Posted on by Adam Fowler

Deploying a network printer via Group Policy is pretty easy. In Group Policy Management Editor, you go to User Configuration > Preferences > Control Panel Settings > Printers and right click to create a new Shared Printer. Configure the options which are pretty straight forward.

Something doesn’t go right though, if you use the option ‘set this printer as the default printer’. The printer won’t actually be set as default for the first login. As it’s set to Update though, this will get fixed next time Group Policy runs.

However, if you have the Action set to ‘Create’, it only gets one chance to set the default printer – at the time of creation. That fails, and it doesn’t get a chance to set the default printer again.

Why does it fail to set the default printer at first logon? You’ll see an event viewer application error like this:

The user ‘HP Printer’ preference item in the ‘Define Printers {XXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}’ Group Policy object did not apply because it failed with error code ‘0x80070709 The printer name is invalid.’ This error was suppressed.

There’s a clear Microsoft Support Article that explains why – in summary, Windows isn’t ready yet to change something on the printer between the time it creates the printer, and then tries to make it default as they’re two separate actions.

However, we can work around this by deploying a registry entry that sets the default printer. This 11 year old article is still correct in that it shows the registry value to change:

HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Windows

REG_SZ - Device
Value - \\adamfowlerit\printer,winspool,Ne04:e


Setting this via registry as an ‘Apply once and do not reapply’ with any other logic matching your printer deployment policy, should end up with the printer as a default on first logon.




This is a problem that’s been around for many years, but the first time I’ve hit it!


Also, if you’re wondering what the ‘Ne04’ part means, it seems to be some sort of counter which goes up for each time you install a printer.

			

	
	




			
				


	

		
How To Grep in PowerShell


				

			Posted on  by Adam Fowler		

			


		

		



              

              
 How to Grep in PowerShell:





 

                           
Here’s some example commands in PowerShell to replicate Grep:


                           
    

                                         
  • Get-Process | Where-Object {$_ | Select-String “foo”} 
    • 

                                         
  • get-process | where ProcessName -like “*foo*
    • 

                                         
  • get-process | findstr foo
    • 

                                         
  • ps *foo*
    • 

                           


 

              

              
Applies To : PowerShell


              

              

              









For those who have lived in the Linux/Unix command line, the ‘grep‘ command is a commonly used way of finding something that you want in a chunk of data.





Øyvind Kallstad did a great writeup of comparing a bunch of ways to use PowerShell instead of grep which is worth reading.





The article covers a bunch of scenarios, and is centered around starting with the ‘grep’ command and working with it. However, there’s the other common use case of running a different command, then piping those results to grep to search for something.





This blogpost was triggered by Janet who asked me this fair question:






https://twitter.com/missjte/status/834132904045342720







https://twitter.com/missjte/status/834133021829664770






As with poor cute cats, there’s more than one way to skin PowerShell.





I had to do some research and asking around on this, because normally I’d filter out the property of the object I was looking at, and work with that. Using the get-process example:





get-process | where ProcessName -like "*foo*





That works, but it’s still a lot clunkier than what a grep user would expect. An easier way would be to use the ‘findstr‘ program (which also has a bunch of useful swtiches):





get-process | findstr foo





I say program because ‘findstr’ is not a PowerShell cmdlet, but it’s still native to Windows and works perfectly fine. It’s case sensitive though, so you need to use -i for case insensitive results.





That’s great for simple stuff, but we’re sort of breaking what PowerShell does. You’re no longer dealing with a standard PowerShell object, so further piping and processing won’t really work.





The ‘proper’ PowerShell way would be to use the ‘Where-Object’ command:





Get-Process | Where-Object {$_ | Select-String "foo"}





A bit longer, but you can shorten ‘Where-Object’ to ‘Where’. Although more involved, it’s good to get into the habit of doing it this way, so when you’re piping this to the next command, it still says as a standard object that can be read and manipulated.’





(Update 24th Feb 2017) As Steve_N points out in the comments section, there’s a much shorter way of doing this:





ps *foo*





That’s it. Many PowerShell commands have inbuilt aliases, including ‘get-process’. You can see what this is with the command ‘get-alias -definition get-process’










This shows that ‘gps’ and ‘ps’ are both aliases to the command  ‘get-process’. You can also create your own aliases with the ‘set-alias‘ command.





The ‘*foo*’ part works because the command assumes the -name switch has been used, which lets you define what criteria to search and show in the ProcessName field. This is the same way that many commands don’t need the -identity switch used, because it’s written to assume you’re going to tell it what identity/username/upn to work with.





This can also be piped to something else, so it’s a winner. It’s less ideal for scripts though, because it’s much harder to read, and you can’t assume that everyone will know the short alias of a full command.





Also note that this isn’t grep related at all, so part of the answer to the original question is that you may not even need grep or select-string as it adds unnecessary overhead of getting data and parsing it, whereas this updated example filters the data as it’s obtained.





(Update ends)





PowerShell isn’t a Linux/Unix command line, but Microsoft have incorporated many of the concepts from bash. If you still can’t bear to use PowerShell on Windows, there’s always the Linux Bash Shell on Windows.





Thanks again to Steve Mclnerey for the grep advice :)

			

	
	




			
				


	

		
Windows 10 – Time To Get On Board


				

			Posted on  by Adam Fowler		

			


		

		
Windows 10 has been publicly available since 29th July 2015. Since then, Microsoft have been encouraging users to upgrade in many ways – consumers had a year window to upgrade from Windows 7/8/8.1 for free, along with Windows Update prompts reminding consumers that they can do so.


There’s always going to be complaints with any new operating system, but the in-place upgrade process has been the best yet from Microsoft. Gone are the days when any IT professional would strongly avoid it, it’s a much more stable and revertable method.


The upgrade has been optional, but we’re now getting much closer to being forced to go Windows 10 (not that I think this is a bad thing). The two big ways this is happening are:


New PCs with Windows 7 or 8.1 are going to be much less common come November 1, 2016. The top OEM vendors won’t be allowed to do this anymore (E.g. Lenovo, HP, Dell). You could still go to a whitebox builder and buy an OEM version of Windows 7, it just won’t be a pre-packaged option anymore. Windows 7 is very old now, and it’s unrealistic to expect Microsoft as well as all the hardware manufacturers to continue supporting it with new drivers.


The other main driver is Intel’s 7th generation of i series chip, Kaby Lake. This has already been released and seen in some laptops, with desktop CPUs due to be released early 2017. Microsoft is drawing a line in the sand and saying there will be no support at all if you’ve got this new CPU. I have yet to get my hands on a device with these new CPUs to try, so it will be interesting to see if anything breaks with this combination of OS and CPU.


Windows 7 has had a very good run, with great reasons; but the vast improvements that have taken us to Windows 10 (not to mention the better security architecture), as well as internal support for cloud services means this is the way of the future.


If you haven’t started the transition to Windows 10 it’s time to get planning, before you hit the above roadblocks and haven’t put the planning and preparation into the change.


 

			

	
	




			
				


	

		
Rolling back from a bad KB Update


				

			Posted on  by Adam Fowler		

			


		

		
Microsoft releases buggy patches now and then (more commonly now sadly).


Today’s stuff up is KB3097877 which breaks a bunch of things, including things like causing Outlook to crash when reading HTML emails.


Best practise is to have a target group from WSUS that these patches go to first, before going company wide – but either way, you’ll want to remove the patch from the affected PCs.


How do you do this? This is my recommended safe approach:


Step 1. Disable the patch in WSUS.

Just do this now, before anyone else gets it. You’re not going to break anything by choosing the ‘Decline’ option on a patch in WSUS. Make sure you do it to each OS version or product you manage (e.g. Windows 7 32 bit, Windows 7 64 bit, Windows 8 32 bit etc).


Step 2. Test uninstalling the patch manually

Before you go nuts and try to fix all the things at once, do a quick test or two. If you manually uninstall the patch, does it successfully uninstall? Reboot and make sure the PC seems happy (check event viewer!). Reboots may take a while doing system state backups and rolling back the patch.


Step 3. – Set WSUS to Uninstall the patch.

It’s a bit counter intuitive to approve a patch to then set it to remove, but that’s how WSUS works. Find the patch by searching for the KB, and once you right click ‘Approve’, you’ll get the option to choose ‘Approved for Removal’. Make sure you’re targeting the correct Computer Group. If you can’t use WSUS, work out how to get your PCs to run a command like this: “wusa /uninstall /kb:3097877 /quiet /norestart” – without the /norestart, they’ll restart :)


Step 4 – Test Windows Update uninstall

Test another PC’s ability to use Windows Updates to uninstall the patch. ‘Checking for updates’ either through the Windows Update GUI or the good old ‘wuauclt /detectnow’ command will do the trick. Similar to Step 2, check it uninstalls and reboot. You can also check C:\Windows\WindowsUpdate.log to make sure it’s happy (this doesn’t apply to Windows 10 as that log doesn’t exist).


Step 5 – Trigger your PCs to check for Windows Updates

Depending on your group policies, Windows Updates will check at certain intervals and may auto download or auto patch. Easiest thing to do is trigger all your PCs to check Windows Updates now. There’s an easy PowerShell way of doing this here, but requires WinRM to be enabled – you should have this on if you want to be able to do a bunch of cool stuff to your PCs. Otherwise, try psexec which will have the same result. This can take a long time to do! Optional component – WOL your PCs first.


Step 6 – Reboot

Now that you’re ready to clean up, test reboot a PC or two and make sure the patch goes away. If that happens, then schedule all your PCs to reboot. You should have a way of doing this already – SCCM can do it well, you can create a once off scheduled task and push that out to PCs, or a bunch of other ways.


Step 7 – Report in WSUS

WSUS has some nice client reporting options. Search for the KB again, right click and choose ‘Status Report’. This is usually not too lagged in it’s information, and you can check to make sure none of your PCs have the update any more. If there’s only a few, it may be easier to manually fix the remainder.


Capture


Happy cleaning up!