May 26

Remove Spam/Phishing Email From All Mailboxes

In 365, you can use Compliance Searches to search and/or remove emails across all of your user’s mailboxes. Compliance searches have replaced the Search-Mailbox cmdlet, which has been deprecated as of April 2020.

Pre-Requisites

You must be a member of the Discovery Management role group or be assigned the Compliance Search management role. Here’s how you can add a user to the Discovery Management group and confirm membership

#Connect to Security and Compliance Center
$UserCredential = Get-Credential
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.compliance.protection.outlook.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic -AllowRedirection
Import-PSSession $Session -DisableNameChecking

#Add User to Discovery Management Group (Replace John Doe with your user)
Add-RoleGroupMember -Identity "Discovery Management" -Member "John Doe"

#Confirm Membership
Get-RoleGroupMember -Identity "Discovery Management"

#Disconnect Session
Remove-PSSession $Session

Function to Search & Remove Spam/Phishing Emails From All Mailboxes

Modify the Search Variables below to suite your needs

Function RemoveMaliciousEmails()
{
	#MODIFY THE BELOW VARIABLES - YOU MUST USE THEM ALL AND OPTIONALLY LEAVE SUBJECT BLANK!!!
	$compSearchName = "MaliciousEmail_$(Get-Date -Format "MMddyyyy_HHmm")"
	$compStartDate = "2020-02-24"  #YYYY-MM-DD
	$compEndDate = "2020-02-25"	   #Must be different than Start Date, if searching "today" make this date tomorrow
	$compFrom = "hacker@phishing.com" #You can use just the domain for a wildcard sender BUT USE WITH CAUTION
	$compSubject = "Phishing Test"  #Use backtick ` to escape special characters in the subject such as a quote ex: Dave shared `"New File`" with you
	#
	#DO NOT MODIFY ANYTHING BELOW HERE
	$UserCredential = Get-Credential
	$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.compliance.protection.outlook.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic -AllowRedirection
	Import-PSSession $Session -DisableNameChecking
	Write-Host "NAME: $($compSearchName)"
	Write-Host "START: $($compStartDate)"
	Write-Host "END: $($compEndDate)"
	Write-Host "FROM: $($compFrom)"
	Write-Host "SUBJECT: $($compSubject)"
	If ($compSubject.Trim() -eq "")
	{
		$ncs = New-ComplianceSearch -Name $compSearchName -ExchangeLocation all -ContentMatchQuery "(c:c)(received=$compStartDate..$compEndDate)(from=$compFrom)"
	}
	Else
	{
		$ncs = New-ComplianceSearch -Name $compSearchName -ExchangeLocation all -ContentMatchQuery "(c:c)(received=$compStartDate..$compEndDate)(from=$compFrom)(subject=`"$compSubject`")"
	}	
	Write-Host "[$(Get-Date -Format G)] Start Compliance Search: $($compSearchName)"
	Write-Host -NoNewLine "[$(Get-Date -Format G)] Searching"
	Start-ComplianceSearch -Identity $compSearchName
	$resultSearch = Get-ComplianceSearch -Identity $compSearchName
	While ($resultSearch.Status -ne "Completed")
	{
		$resultSearch = Get-ComplianceSearch -Identity $compSearchName
		Write-Host -NoNewLine "."
		Start-Sleep -s 5
	}
	If (($resultSearch.Items -le 0) -OR ([string]::IsNullOrWhiteSpace($resultSearch.SuccessResults)))
	{
		Write-Host "`n[$(Get-Date -Format G)] Search completed with 0 successful results, Invalid Search Quitting!" -ForegroundColor Red
		Remove-PSSession $Session
		Return
	}
	Elseif ($resultSearch.Items -ge 500)
	{
		Write-Host "`n[$(Get-Date -Format G)] Search Completed with $($resultSearch.Items) successful results, you will need to run mulitple times!" -ForegroundColor Yellow
	}
	Else 
	{
		Write-Host "`n[$(Get-Date -Format G)] Search Completed with $($resultSearch.Items) successful results"
	}
	Write-Host "[$(Get-Date -Format G)] Create Preview and Export of Results"
	Write-Host -NoNewLine "[$(Get-Date -Format G)] Processing"
	$ncsa = New-ComplianceSearchAction -SearchName $compSearchName -Preview
	$resultPreview = Get-ComplianceSearchAction -Identity "$($compSearchName)_Preview"
	$ncsa = New-ComplianceSearchAction -SearchName $compSearchName -Export -ExchangeArchiveFormat SinglePst -Format FxStream
	$resultExport = Get-ComplianceSearchAction -Identity "$($compSearchName)_Export"
	While ($resultPreview.Status -ne "Completed" -AND $resultExport.Status -ne "Completed")
	{
		$resultPreview = Get-ComplianceSearchAction -Identity "$($compSearchName)_Preview"
		$resultExport = Get-ComplianceSearchAction -Identity "$($compSearchName)_Export"
		#Write-Host "[$(Get-Date -Format G)] Processing..."
		Write-Host -NoNewLine "."
		Start-Sleep -s 5
	}
	Write-Host "`n[$(Get-Date -Format G)] Preview and Export successfully created"
	Write-Host "[$(Get-Date -Format G)] View Results at https://protection.office.com/" 
	Write-Host "[$(Get-Date -Format G)] Preview: Search -> Content Search -> Searches tab -> $($compSearchName)"
	Write-Host "[$(Get-Date -Format G)] Export:  Search -> Content Search -> Exports tab -> $($compSearchName)_Export"
	Write-Host "[$(Get-Date -Format G)] Start Purging emails"
	Write-Host -NoNewLine "[$(Get-Date -Format G)] Purging"
	$ncsa = New-ComplianceSearchAction -SearchName $compSearchName -Purge -PurgeType HardDelete -Confirm:$False
	$resultPurge = Get-ComplianceSearchAction -Identity "$($compSearchName)_Purge"
	While ($resultPurge.Status -ne "Completed")
	{
		$resultPurge = Get-ComplianceSearchAction -Identity "$($compSearchName)_Purge"
		Write-Host -NoNewLine "."
		Start-Sleep -s 5
	}
	Write-Host "`n[$(Get-Date -Format G)] Purge complete"
	$confirmDelete = Read-Host "Delete Compliance Search, Preview, Export, and Purge? [Y/N]"
	If ($confirmDelete -eq 'Y') 
	{
		Write-Host "[$(Get-Date -Format G)] Deleting Compliance Search, Preview, Export, and Purge..."
		Remove-ComplianceSearchAction -Identity "$($compSearchName)_Preview" -Confirm:$False
		Remove-ComplianceSearchAction -Identity "$($compSearchName)_Export" -Confirm:$False
		Remove-ComplianceSearchAction -Identity "$($compSearchName)_Purge" -Confirm:$False
		Remove-ComplianceSearch -Identity $compSearchName -Confirm:$False
	}
	Write-Host "[$(Get-Date -Format G)] Removing Powershell Session..."
	Remove-PSSession $Session
}
RemoveMaliciousEmails
Jun 19

Clean No-Intro 2018 ROM Set

IMPORTANT UPDATE 12/06/2020 – This is updated for v5.3!!!

Overview

I did not want my RetroPie to be cluttered with ROMs I did not want. I only wanted the original games, I did not want duplicates or revisions, and I only wanted games in English so I could read the on screen text. For example, the NES has about 716 officially licensed games and the No-Intro 2018 ROM set has about 2,748 games in it. I wanted to reduce that number closer to the officially licensed games count to save space and reduce the amount of ROMs I would have to scroll through each time. I decided to create the below powershell scripts based on the No-Intro 2018 ROM set to accomplish my goal, which I’ll explain each step in detail.

http://sigkillit.com/wp-content/uploads/2020/12/Clean-NoIntro2018_v53.zip

You can also download the older archived scripts, but I HIGHLY RECOMMEND sticking with latest scripts!

The No-Intro 2018 ROM set includes the following game systems that are compatible with RetroPie/Emulationstation:

  • Atari: 2600
  • Atari: 5200
  • Atari: 7800
  • Atari: Jaguar
  • Atari: Lynx
  • Bandai: WonderSwan
  • Bandai: WonderSwan Color
  • Coleco: ColecoVision
  • GCE: Vectrex
  • Microsoft: MSX
  • NEC: PC Engine / TurboGrafx 16
  • Nintendo: Family Computer Disk System
  • Nintendo: Game Boy
  • Nintendo: Game Boy Advance
  • Nintendo: Game Boy Color
  • Nintendo: Nintendo 64
  • Nintendo: Nintendo Entertainment System
  • Nintendo: Super Nintendo Entertainment System
  • Nintendo: Virtual Boy
  • SNK: Neo Geo Pocket
  • SNK: Neo Geo Pocket Color
  • Sega: 32X
  • Sega: Game Gear
  • Sega: Master System / Mark III
  • Sega: Mega Drive / Genesis
  • Sega: SG-1000

The No-Intro ROMs follow a pretty standardized naming scheme such as:

  • Battletoads (Europe).zip
  • Battletoads (Japan).zip
  • Battletoads (USA).zip
  • Captain Skyhawk (Europe).zip
  • Captain Skyhawk (USA) (Rev A).zip
  • Captain Skyhawk (USA).zip

As you can see, the base name of the ROM is followed by detailed descriptions such as the region released, revision, language, etc which is contained in parentheses. The script is based on using this naming scheme to get unique ROM base names by grabbing everything to the left of the first parentheses. It then further analyzes the results based on filters for each game system, which decide the ROM to keep. The default filters are set to prefer the USA version and the earliest revision (On older gaming systems, later revisions usually just made changes like the sprite used for a charactor or the color and not necessarily “bug” fixes like more modern game systems). Using the examples above, the script would keep “Battletoads (USA).zip) and “Captain Skyhawk (USA).zip”. See below for further details on using the scripts.

1_Clean-NoIntroRoms.ps1

See supplemental scripts:

Global Variable Definitions:

  • $global:scriptName
    • Name of the Script, defaults to full name of the script
  • $global:scriptVersion
    • Script Version
  • $global:romNoIntroDirectory
    • Source No-Intro 2018 Game System ROM Directory, each game system should have a folder in here that matches in $global:romNoIntroHashTable. Default is “C:\Games\No-Intro 2018\”
  • $global:romCleanDirectory
    • Destination directory to copy No-Intro 2018 ROMs to, each game system will have a folder in here that matches in $global:romNoIntroHashTable (Default is the RetroPie game system directory names). This way it never modifies the ROMs in the source directory. Default is “C:\Games\RetroPie\roms\”
  • $global:romCleanDirectoryPurge
    • Remove each game system directory in $global:romCleanDirectory so it does not exist before copying the ROMs. Default is $true
  • $global:romOverWrite
    • Overwrite existing ROMs in each game system directory in $global:romCleanDirectory if the game system directory exists. When $false, it will not copy any ROMs into the game system directory if it exists. When $true, it will copy and overwrite ROMs into the game system directory if it exists, but will not remove any additional ROMs or files that exist. Default is $false
  • $global:romFilterDirectory
    • Directory that stores the “filters” for each game system. The filters are .ps1 files named using the RetroPie directory name defined in $global:romNoIntroHashTable. I have customized the filters for each system and included them by default. You can modify them to your liking or if they do not exist, the script will generate a generic filter for each system. Default is a “filters” folder in the same directory as the script
  • $global:romRetropieDirectory
    • Location of the ROM directory on the RetroPie, which is used if you want the script to auto copy the ROMs to your RetroPie. Default is “\\retropie\roms\”
  • $global:romRetropieDirectoryPurge
    • Purge each game system’s ROM directory before copying ROMs to the RetroPie. Default is $true
  • $global:romRetropieUpload
    • Auto copy the ROMs from $global:romCleanDirectory to $global:romRetropieDirectory. Default is $true
  • $global:logFile
    • Log file output of the entire run and additional log files are placed in the same directory. Default is a “logs” directory in the same directory as the script with a log name that matches the script’s base name. Ex: .\logs\1_Clean-NoIntroRom.log
  • $global:logOverwrite
    • Overwrite the log files. Default is $true
  • $global:logVerbose
    • Show the full details of how it decides which ROMs to keep. This should only be used for debugging purposes and will increase the run time of the script. Default is $false
  • $global:romNoIntroHashTable
    • Hashtable setting each game systems directory name on RetroPie and in the No-Intro 2018 complete ROM set. Default directory names are used for each

Filter File Variable Definitions:

I have customized and included these since v5.0 and they are located in .\filters\<gamesystem>.ps1. If you want customize which ROMs are kept for each system, this is what you want to change.

  • $romUnzip
    • Unzip ROMs for emulators that don’t support .zip. $false will NOT unzip them and $true will unzip them
  • $romsDoNotDelete
    • Any exact ROM name (including extension) in this list will NOT be deleted. ROMs listed here will not be deleted even in they match a value in $romsExceptions or they are NOT the Primary ROM via $romSortOrder
    • Ex: “Castlevania (USA) (Rev A).zip”
  • $romsExceptions
    • Any ROMs containing these strings or exact ROM name will be removed unless the exact ROM name is in $romsDoNotDelete
  • $romsSortOrder
    • Preferred sorting order to pick 1 Primary ROM to keep when evaluating duplicates. Uses regular expressions to define the preferred sorting order based on language, and if there are no matches it will determine the primary ROM by alphabetical sort. ROMs in $romsExceptions will not be deleted even if it is not determined to be the Primary ROM
#Clean-NoIntroRoms.ps1
#5.3
# -Re-Added Function Get-NoIntroDefaultFilters which was accidently removed from version 5.2
# -Renamed "config" to "filter" Main Function Section 1 Log
# -Added Changelog in Comments
#5.2
# -Added variables for Write-Out Colors and Updated to Write-Out v1.3
# -Updated Filters for scraping
#5.1
# -Fixed minor bugs and updating filters for scraping
#5.0
# -Rewrote the code to be modular by importing filters for each game system instead of using separate scripts
# -Added variables to define where to store game system filters and logs
# -Replaced $MyInvocation.MyCommand.Name with $PSCommandPath which requires Powershell 3 or greater
# -Replaced Log-Write with Write-Out function which requires Powershell 4 or greater
# -Added Atari Jaguar because it can run on PI4 with proper customization
# -Fixed bug with romsDoNotDelete
# -Fixed bug where exceptions were not removed from uniqueRom causing it to still evaluate them for duplicates
$global:scriptName = $PSCommandPath
$global:scriptVersion = "v5.3"
$global:romNoIntroDirectory = "C:\Games\Complete ROM Sets\No-Intro 2018\"
$global:romCleanDirectory = "C:\Games\RetroPie\roms\"
$global:romCleanDirectoryPurge = $true
$global:romOverWrite = $false
$global:romFilterDirectory = (Split-Path -Path $PSCommandPath -Parent) + "\filters\"
$global:romRetropieDirectory = "\\retropie\roms\"
$global:romRetropieDirectoryPurge = $true
$global:romRetropieUpload = $true
$global:logFile = (Split-Path -Path $PSCommandPath -Parent) + "\logs\" + (Split-Path -Path $PSCommandPath.Replace(".ps1",".log") -Leaf)
$global:logOverwrite = $true
$global:logVerbose = $false
$global:romNoIntroHashTable = [ordered]@{
	#RetroPieDirName = No-IntroDirName
	"atari2600" = "Atari - 2600" 
	"atari5200"  = "Atari - 5200"
	"atari7800" = "Atari - 7800 [Headered]"
	"atarijaguar" = "Atari - Jaguar"
	"atarilynx" = "Atari - Lynx [Headered]"
	"atarist" = "Atari - ST"
	"coleco" = "Coleco - ColecoVision"
	"fds" = "Nintendo - Family Computer Disk System [headered]"
	"gamegear" = "Sega - Game Gear"
	"gb" = "Nintendo - Game Boy"
	"gba" = "Nintendo - Game Boy Advance"
	"gbc" = "Nintendo - Game Boy Color"
	"mastersystem" = "Sega - Master System - Mark III"
	"megadrive" = "Sega - Mega Drive - Genesis"
	"msx" = "Microsoft - MSX"
	"n64" = "Nintendo - Nintendo 64"
	"nes" = "Nintendo - Nintendo Entertainment System [headered]"
	"ngp" = "SNK - Neo Geo Pocket"
	"ngpc" = "SNK - Neo Geo Pocket Color"
	"pcengine" = "NEC - PC Engine - TurboGrafx 16"
	"sega32x" = "Sega - 32X"
	"sg-1000" = "Sega - SG-1000"
	"snes" = "Nintendo - Super Nintendo Entertainment System"
	"vectrex" = "GCE - Vectrex"
	"virtualboy" = "Nintendo - Virtual Boy"
	"wonderswan" = "Bandai - WonderSwan"
	"wonderswancolor" = "Bandai - WonderSwan Color"
}
#Output Color Variables
$global:colorMain = "Green"
$global:colorSub1 = "DarkGreen"
$global:colorSub2 = "Green"
$global:colorSub3 = "DarkRed"
$global:colorOk = "Green"
$global:colorWarn = "Yellow"
$global:colorError = "Red"



Function Main()
{	
	#Powershell 4 or greater is required
	If (!($PSVersionTable.PSVersion.Major -ge 4) -OR ($PSVersionTable.PSVersion -eq $null))
	{
		Write-Host -ForegroundColor $global:colorError "Powershell v4 or greater is required, quitting script!"
		Exit
	}
	
	#Create Log File
	If ($global:logOverwrite -eq $true)
	{
		Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile -Overwrite
	}
	Else 
	{
		Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
	}
	Write-Out -ForegroundColor $global:colorMain -Text "SECTION 1: INITIALIZE" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Script Name             : $($global:scriptName)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Version:                : $($global:scriptVersion)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM No-Intro Dir        : $($global:romNoIntroDirectory)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM Clean Dir           : $($global:romCleanDirectory)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM Clean Dir Purge     : $($global:romCleanDirectoryPurge)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM Overwrite           : $($global:romOverWrite)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM Filter Dir          : $($global:romFilterDirectory)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM Retropie Dir        : $($global:romRetropieDirectory)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM Retropie Dir Purge  : $($global:romRetropieDirectoryPurge)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM Upload to Retropie  : $($global:romRetropieUpload)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Log File                : $($global:logFile)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Log Overwrite           : $($global:logOverwrite)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Log Verbose             : $($global:logVerbose)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Description             : Cleanup 2018 No-Intro ROMs" -LogFile $global:logFile
	$tStart = (Get-Date)
	Write-Out -ForegroundColor $global:colorMain -Text "* Start Time              : $($tStart)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile

	$tLog = $global:logFile.Replace(".log","_Exceptions.log")
	Write-Out -PreLine 1 -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "SECTION 2: BUILD LIST OF UNIQUE EXCEPTIONS FILTERS FOR NO-INTRO ROMS" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Log File                : $($tLog)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Log Overwrite           : $($global:logOverwrite)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Description             : Use No-Intro Naming Scheme to Build a List of Unique Exception Filters Used to Remove" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "*                           Unwanted ROMs For Each System Ex: (Beta), (Proto),(Japan)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
	$ArgumentHash = @{
		Source = $global:romNoIntroDirectory
		HashTable = $global:romNoIntroHashTable
		Log = $tLog 
		LogOverwrite = $global:logOverwrite
	}
	Get-NoIntroRomsExceptions @ArgumentHash

	Write-Out -PreLine 1 -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "SECTION 3: CLEANUP NO-INTRO ROMS" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Log File                : Per Game System" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Log Overwrite           : $($global:logOverwrite)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Description             : Use filters to copy No-Intro 2018 ROMs from Source to clean directory" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
	ForEach ($key in $global:romNoIntroHashTable.keys)
	{
		$tLog = "$((Split-Path -Path $global:logFile -Parent))\$($key).log"
		$tFilters = "$($global:romFilterDirectory)$($key).ps1"
		$tSource = "$($global:romNoIntroDirectory)$($global:romNoIntroHashTable[$key])"
		$tDestination = "$($global:romCleanDirectory)$($key)"
		Write-Out -PreLine 1 -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorMain -Text "PROCESS: $key" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorSub1 -Text "ROM Description         : $($global:romNoIntroHashTable[$key])" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorSub1 -Text "ROM Filters             : $($tFilters)" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorSub1 -Text "ROM Source Dir          : $($tSource)" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorSub1 -Text "ROM Clean Dir           : $($tDestination)" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorSub1 -Text "ROM Clean Dir Purge     : $($global:romCleanDirectoryPurge)" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorSub1 -Text "ROM Overwrite           : $($global:romOverWrite)" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorSub1 -Text "Log File                : $($tLog)" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorSub1 -Text "Log Overwrite           : $($global:logOverwrite)" -LogFile $global:logFile
		Write-Out -ForegroundColor $global:colorSub1 -Text "Log Verbose             : $($global:logVerbose)" -LogFile $global:logFile
		If ($global:romCleanDirectoryPurge)
		{
			If(Test-Path -LiteralPath "$($tDestination)")
			{
				Write-Out -ForegroundColor $global:colorSub1 -Text "ROM Clean Dir Exists, Purge Before Processing" -LogFile $global:logFile
				Write-Out -ForegroundColor DarkRed -Text "Purge: $($tDestination)" -LogFile $global:logFile
				Remove-Item "$($tDestination)" -Recurse -Force -Confirm:$false
			}
			Else
			{
				Write-Out -ForegroundColor $global:colorWarn -Text "ROM Clean Dir Does Not Exist, Skip Purge Before Processing" -LogFile $global:logFile
			}
		}
		$ArgumentHash = @{
			System = $key
			Description = $global:romNoIntroHashTable[$key] 
			Source = $tSource 
			Destination = $tDestination 
			Overwrite = $global:romOverWrite
			Filters = $tFilters 
			Log = $tLog 
			LogOverwrite = $global:logOverwrite 
			LogVerbose = $global:logVerbose
		}
		Clean-NoIntroRoms @ArgumentHash
		Write-Out -ForegroundColor $global:colorSub1 -Text "Processing $($key) Complete" -LogFile $global:logFile
	}	
	
	Write-Out -PreLine 1 -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "SECTION 4: END RESULTS" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM Retropie Dir        : $($global:romRetropieDirectory)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM Retropie Dir Purge  : $($global:romRetropieDirectoryPurge)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* ROM Upload to Retropie  : $($global:romRetropieUpload)" -LogFile $global:logFile
	If ($global:romRetropieUpload)
	{
		#Loop Each ROM Directory and Copy to RetroPie
		ForEach ($key in $global:romNoIntroHashTable.keys)
		{
			$tSource = "$($global:romCleanDirectory)$($key)"
			$tDestination = "$($global:romRetropieDirectory)$($key)"
			Write-Out -PreLine 1 -ForegroundColor $global:colorMain -Text "* $($key)" -LogFile $global:logFile
			Write-Out -ForegroundColor $global:colorMain -Text "* =======================================================" -LogFile $global:logFile
			If (Test-Path -LiteralPath "$($tSource)")
			{
				Write-Out -ForegroundColor $global:colorMain -Text "* Source Exist: $($tSource)" -LogFile $global:logFile
				If (Test-Path -LiteralPath "$($tDestination)")
				{
					Write-Out -ForegroundColor $global:colorMain -Text "* Dest Exist: $($tDestination)" -LogFile $global:logFile
					If ($global:romRetropieDirectoryPurge)
					{
						Write-Out -ForegroundColor $global:colorError -Text "* Purge: $($tDestination)" -LogFile $global:logFile
						Get-ChildItem "$($tDestination)" -Recurse | Remove-Item -Force -Confirm:$false
					}
					Write-Out -ForegroundColor $global:colorMain -Text "* Copy ROMs from '$($tSource)' to '$($tDestination)'" -LogFile $global:logFile
					Copy-Item -Path "$($tSource)\*" -Destination "$($tDestination)" -Recurse -Force
				}
				Else 
				{
					Write-Out -ForegroundColor $global:colorWarn -Text "* Dest NOT Exist: $($tDestination)" -LogFile $global:logFile
					Write-Out -ForegroundColor $global:colorMain -Text "* Create Folder: $($tDestination)" -LogFile $global:logFile
					If (New-Item -ItemType Directory -Path $tDestination -Force)
					{
						Write-Out -ForegroundColor $global:colorMain -Text "* Copy ROMs from '$($tSource)' to '$($tDestination)'" -LogFile $global:logFile
						Copy-Item -Path "$($tSource)\*" -Destination "$($tDestination)" -Recurse -Force
					}
					Else
					{
						Write-Out -ForegroundColor $global:colorError -Text "* ERROR Creating Destination No ROMs Copied!" -LogFile $global:logFile
					}
				}
			}
			Else
			{
				Write-Out -ForegroundColor $global:colorError -Text "* Source NOT Exist: $($tSource)" -LogFile $global:logFile
				Write-Out -ForegroundColor $global:colorError -Text "* ERROR CANNOT COPY!!!" -LogFile $global:logFile
			}
		}
	}
	$tEnd = (Get-Date)
	Write-Out -ForegroundColor $global:colorMain -Text "* End Time                : $($tEnd)" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "* Elapsed Time            : $([math]::Round(($tEnd-$tStart).totalseconds, 2)) seconds" -LogFile $global:logFile
	Write-Out -ForegroundColor $global:colorMain -Text "****************************************************************" -LogFile $global:logFile
}



Function Get-NoIntroRomsExceptions()
{
	Param
	(	
		[Parameter(Mandatory=$true)]
		[Alias("Source")] 
		[string]$RomDir,
		
		[Parameter(Mandatory=$true)]
		[Alias("HashTable")] 
		[hashtable]$RomHashTable,
		
		[Parameter(Mandatory=$true)]
		[Alias("Log")]
		[string]$LogFile,
		
		[Parameter(Mandatory=$false)]
		[string]$LogOverwrite = $false
	)
	If ($LogOverwrite -eq $true)
	{
		Write-Out -ForegroundColor $global:colorMain -Text " " -LogFile $LogFile -Overwrite
	}
	Else 
	{
		Write-Out -ForegroundColor $global:colorMain -Text " " -LogFile $LogFile
	}
	
	$colUniqueRomsAll = @()
	ForEach ($key in $RomHashTable.keys)
	{
		#Process Current ROM Directory
		Write-Out -PreLine 1 -ForegroundColor $global:colorMain -Text "PROCESS: $($RomDir)$($RomHashTable[$key])" -LogFile $LogFile
		Write-Out -ForegroundColor $global:colorMain -Text "=======================================================" -LogFile $LogFile
		
		#Confirm No-Intro ROM Directory Exists
		If(!(Test-Path -LiteralPath "$($RomDir)$($RomHashTable[$key])"))
		{
			#Directory Does Not Exist
			Write-Out -ForegroundColor $global:colorError -Text "ERROR: ROM Directory Does Not Exist!!!" -LogFile $LogFile
		}
		Else 
		{
			#Directory Exists
			Write-Out -ForegroundColor $global:colorSub1 -Text "Directory Exists: Continue Processing" -LogFile $LogFile
			$colRoms = @()
			$colUniqueRoms = @()
			$colRoms = Get-ChildItem -LiteralPath "$($RomDir)$($RomHashTable[$key])"
			$colRoms | ForEach-Object `
			{
				$colUniqueRoms += '(' + $_.basename.split("(",2)[1]
			}
			$colUniqueRoms = $colUniqueRoms | Sort-Object -Unique
			$colUniqueRomsAll += $colUniqueRoms
			Write-Out -ForegroundColor $global:colorMain -Text "Found $($colUniqueRoms.Count) Potential Exceptions to Filter:" -LogFile $LogFile
			ForEach ($name in $colUniqueRoms)
			{
				Write-Out -ForegroundColor $global:colorSub1 -Text "$($name)" -LogFile $LogFile
			}
		}
	}
	Write-Out -PreLine 1 -ForegroundColor $global:colorMain -Text "PROCESS: Combined Unique Exceptions" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorMain -Text "=======================================================" -LogFile $LogFile
	$colUniqueRomsAll = $colUniqueRomsAll | Sort-Object -Unique
	Write-Out -ForegroundColor $global:colorMain -Text "Found $($colUniqueRomsAll.Count) Combined Potential Exceptions to Filter:" -LogFile $LogFile
	ForEach ($name in $colUniqueRomsAll)
	{
		Write-Out -ForegroundColor $global:colorSub1 -Text "$($name)" -LogFile $LogFile
	}
}



Function Clean-NoIntroRoms()
{
	Param
	(
			
		[Parameter(Mandatory=$true)]
		[Alias("System")]
		[string]$GameSystem,
		
		[Parameter(Mandatory=$true)]
		[Alias("Description")]
		[string]$GameDescription,
	
		[Parameter(Mandatory=$true)]
		[Alias("Source")]
		[string]$RomSource,
		
		[Parameter(Mandatory=$true)]
		[Alias("Destination")]
		[string]$RomDestination,
		
		[Parameter(Mandatory=$true)]
		[Alias("Overwrite")]
		[bool]$RomOverwrite,
		
		[Parameter(Mandatory=$true)]
		[Alias("Filters")]
		[string]$RomFilters,
		
		[Parameter(Mandatory=$true)]
		[Alias("Log")]
		[string]$LogFile,
		
		[Parameter(Mandatory=$false)]
		[bool]$LogOverwrite = $true,
		
		[Parameter(Mandatory=$false)]
		[bool]$LogVerbose = $false		
	)
	$countTotal  = 0
	$countUnique = 0
	$countExceptionsDeleted = 0
	$countDuplicatesDeleted = 0
	$countNonDuplicatesDeleted = 0

	#Create Log File
	If ($LogOverwrite -eq $true)
	{
		Write-Out -ForegroundColor $global:colorMain -Text "$($GameSystem.ToUpper()) (STEP 1/6): Processing Information" -LogFile $LogFile -Overwrite
	}
	Else 
	{
		Write-Out -ForegroundColor $global:colorMain -Text "$($GameSystem.ToUpper()) (STEP 1/6): Processing Information" -LogFile $LogFile
	}
	Write-Out -ForegroundColor $global:colorMain -Text "================================================================" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "Game System             : $($GameDescription)" -LogFile $LogFile
	$timeStart = (Get-Date)
	Write-Out -ForegroundColor $global:colorSub1 -Text "Start Time              : $($timeStart)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROM Source Dir          : $($RomSource)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROM Clean  Dir          : $($RomDestination)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROMs Overwrite          : $($RomOverwrite)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROM Filters             : $($RomFilters)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "Log File                : $($LogFile)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "Log Overwrite           : $($LogOverwrite)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "Log Verbose             : $($LogVerbose)" -LogFile $LogFile
	If(!(Test-Path -LiteralPath $RomSource))
	{
		Write-Out -ForegroundColor $global:colorError -Text "ERROR: ROM Source Directory Does Not Exist - Stop Processing!" -LogFile $LogFile
		Return
	}
	If(!(Test-Path -LiteralPath $RomFilters))
	{
		#Set Defaults For: $romUnzip, $romsDoNotDelete, $romsExceptions, $romsSortOrder
		Write-Out -ForegroundColor $global:colorWarn -Text "WARNING: ROM Filters File Does Not Exist, Set Default Values - Continue Processing!" -LogFile $LogFile
		Write-Out -ForegroundColor $global:colorWarn -Text "$($RomFilters): Write `$romUnzip Default" -LogFile $LogFile
		Write-Out -ForegroundColor $global:colorWarn -Text "$($RomFilters): Write `$romsDoNotDelete Default" -LogFile $LogFile
		Write-Out -ForegroundColor $global:colorWarn -Text "$($RomFilters): Write `$romsExceptions Default" -LogFile $LogFile
		Write-Out -ForegroundColor $global:colorWarn -Text "$($RomFilters): Write `$romsSortOrder Default" -LogFile $LogFile
		$DefaultFilters = Get-NoIntroDefaultFilters	(Split-Path -Path $RomFilters -Leaf)
		Write-Out -ForegroundColor $global:colorWarn -Text "Write to File: $($RomFilters)" -LogFile $LogFile
		$nf = New-Item -Path $RomFilters -type file -force -Value $DefaultFilters
	}

	#Set Custom Variables to Cleanup ROMs for Each System by Dot Sourcing($romUnzip, $romsDoNotDelete, $romsExceptions, $romsSortOrder)
	Write-Out -ForegroundColor $global:colorSub1 -Text "Import Variables from $($RomFilters)" -LogFile $LogFile
	. "$($RomFilters)"
	Write-Out -ForegroundColor $global:colorMain -Text "ROM Exceptions to Keep" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorMain -Text "-------------------------------------------------------" -LogFile $LogFile
	If (($romsDoNotDelete.Count -le 0) -OR $romsDoNotDelete -eq $null)
	{
			Write-Out -ForegroundColor $global:colorWarn -Text "None" -LogFile $LogFile
	}
	Else 
	{
		ForEach ($rom in $romsDoNotDelete)
		{
			Write-Out -ForegroundColor $global:colorSub1 -Text "$($rom)" -LogFile $LogFile
		}
	}
	
	Write-Out -ForegroundColor $global:colorMain -Text "Delete ROMs Containing" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorMain -Text "-------------------------------------------------------" -LogFile $LogFile
	If (($romsExceptions.Count -le 0) -OR $romsExceptions -eq $null)
	{
			Write-Out -ForegroundColor $global:colorWarn -Text "None" -LogFile $LogFile
	}
	Else 
	{
		ForEach ($rom in $romsExceptions)
		{
			Write-Out -ForegroundColor $global:colorSub1 -Text "$($rom)" -LogFile $LogFile
		}
	}
	
	Write-Out -ForegroundColor $global:colorMain -Text "ROM Sort Order" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorMain -Text "-------------------------------------------------------" -LogFile $LogFile
	ForEach ($rom in $romsSortOrder)
	{
		Write-Out -ForegroundColor $global:colorSub1 -Text "$($rom)" -LogFile $LogFile
	}
	

	
	#Get List of ROMs and Unique ROMs in the original No-Intro 2018 ROM set
	Write-Out -ForegroundColor $global:colorMain -Text "$($GameSystem.ToUpper()) (STEP 2/6): Get All No-Intro ROMs and Unique ROMs to Process" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorMain -Text "================================================================" -LogFile $LogFile	
	[System.Collections.ArrayList]$colRoms = @()
	(Get-ChildItem -LiteralPath $RomSource).ForEach({$obj = [PSCustomObject]@{ FullName = $_.FullName; Name = $_.Name};$colRoms.add($obj)|out-null})
	$colUniqueRoms = @()
	$colUniqueRoms = $colRoms | ForEach-Object { $_.Name.split("(")[0] } | Sort-Object -Unique
	
	#Count Total Original Source ROMs and Original Source Unique ROMs
	$countTotal = $colRoms.count
	$countUnique = $colUniqueRoms.count
	Write-Out -ForegroundColor $global:colorSub1 -Text "Total ROMs: $($countTotal)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "Total Unique ROMs: $($countUnique)" -LogFile $LogFile



	#Remove Exceptions - Further Speeds Things Up By Not Having to Process As Much
	Write-Out -ForegroundColor $global:colorMain -Text "$($GameSystem.ToUpper()) (STEP 3/6): Remove Exceptions" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorMain -Text "================================================================" -LogFile $LogFile
	If (($romsExceptions.Count -le 0) -OR $romsExceptions -eq $null)
	{
			Write-Out -ForegroundColor $global:colorWarn -Text "None" -LogFile $LogFile
	}
	Else 
	{
		#Remove Exception from colRoms Array
		For ($i=0; $i -lt $colRoms.Count; $i++)
		{
			ForEach ($romException in $romsExceptions) 
			{
				#Check If Current ROM Matches Exception
				If ($colRoms[$i].Name -match [regex]::escape($romException))
				{
					#Exception Found (Flag Delete)
					#Special Case Do Not Delete ROMs Check
					$romDND = $false
					If ($romsDoNotDelete.Count -gt 0)
					{
						ForEach ($doNotDelete in $romsDoNotDelete)
						{
							If ($doNotDelete -eq $colRoms[$i].Name)
							{
								#Found Special Case (Flag NOT Delete)
								$romDND = $true
								break
							}
						}
					}
					#Check to Delete or Not
					If ($romDND -eq $true)
					{
						Write-Out -ForegroundColor $global:colorSub1 -Text "DO NOT DELETE EXCEPTON (Special Case): $($colRoms[$i].Name)" -LogFile $LogFile
					}
					Else
					{
						#Delete ROM
						Write-Out -ForegroundColor DarkRed -Text "DELETE EXCEPTION '$($romException)': $($colRoms[$i].Name)" -LogFile $LogFile
						$colRoms.RemoveAt($i)
						$i--
						$countExceptionsDeleted++
					}
					break
				}
			}
		}
	}
	#Update Unique ROMs to Remove Exceptions
	$colUniqueRoms = $colRoms | ForEach-Object { $_.Name.split("(")[0] } | Sort-Object -Unique

	
	
	#Identify Primary ROM to Keep and Remove Duplicates
	Write-Out -ForegroundColor $global:colorMain -Text "$($GameSystem.ToUpper()) (STEP 4/6): Identify Primary ROM and Remove Duplicates" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorMain -Text "================================================================" -LogFile $LogFile
	ForEach ($uniqueRom in $colUniqueRoms)
	{
		#Determine Duplicates For Each Unique ROM Name
		$colDuplicateRoms = @()
		$colDuplicateRoms = $colRoms.Name -match ("^"+[regex]::escape($uniqueRom)+"\(")
		Write-Out -ForegroundColor $global:colorMain -Text "PROCESS: '$($uniqueRom)'" -LogFile $LogFile
		If ($colDuplicateRoms.count -gt 1)
		{
			#Determine Primary ROM In Each Duplicate ROM Set By Matching Sorting Preference
			:regloop ForEach ($sortRegex in $romsSortOrder )
			{
				#Process ROMS That Match A Sorting Order
				$primaryRom = ""
				$colRomsToSort = @()
				[array]$colRomsToSort = $colDuplicateRoms -match $sortRegex
				$colRomsToSort = $colRomsToSort | Sort-Object
				If ($colRomsToSort.count -ge 1)
				{
					#Find Primary ROM To Keep
					If ($LogVerbose -eq $true)
					{
						Write-Out -ForegroundColor $global:colorSub1 -Text "TEST REGEX ($($colRomsToSort.count) Matches): $($sortRegex)" -LogFile $LogFile
					}
					For($i = $colRomsToSort.GetLowerBound(0); $i -le $colRomsToSort.GetUpperBound(0); $i++) 
					{
						$matchRomException = $false
						If ($LogVerbose -eq $true)
						{
							Write-Out -ForegroundColor $global:colorSub1 -Text "Primary ROM Found (Match $($i + 1) of $($colRomsToSort.GetUpperBound(0) + 1)): $($colRomsToSort[$i])" -LogFile $LogFile
						}
						#Check Primary ROM For Exceptions
						ForEach ($romException in $romsExceptions) 
						{
							If ($colRomsToSort[$i] -match [regex]::escape($romException))
							{
								$matchRomException = $true
								If ($LogVerbose -eq $true)
								{
									Write-Out -ForegroundColor $global:colorSub1 -Text "Primary ROM Exception Found (Match $($i+1) of $($colRomsToSort.count): $($romException)" -LogFile $LogFile
								}
								break
							}
						}
						#Set Primary ROM 
						If ($matchRomException -eq $false)
						{
							$primaryRom = $colRomsToSort[$i]
							break regloop
						}
					}
				}
				Else 
				{
					If ($LogVerbose -eq $true)
					{
						Write-Out -ForegroundColor $global:colorSub1 -Text "SKIP REGEX (No Matches): $($sortRegex)" -LogFile $LogFile
					}
				}
			}
			
			#Display Primary ROM Result
			If ($primaryRom -eq "")
			{
				If ($LogVerbose -eq $true)
				{
					Write-Out -ForegroundColor $global:colorSub1 -Text "Primary ROM Set: No Primary Found!" -LogFile $LogFile
				}
			}
			Else
			{
				If ($LogVerbose -eq $true)
				{
					Write-Out -ForegroundColor $global:colorSub1 -Text "Primary ROM Set: $($primaryRom)" -LogFile $LogFile
				}
			}
			
			#Remove All Duplicate ROMS Except Primary ROM
			ForEach ($duplicateRom in $colDuplicateRoms)
			{
				#Primary ROM Found
				If ($duplicateRom -eq $primaryRom)
				{
					#Keep Primary ROM
					Write-Out -ForegroundColor $global:colorSub1 -Text "KEEP(Primary): $($duplicateRom)" -LogFile $LogFile
				}
				Else
				{
					#Special Case Do Not Delete ROMs Check
					$romDND = $false
					If ($romsDoNotDelete.Count -gt 0)
					{
						ForEach ($doNotDelete in $romsDoNotDelete)
						{
							If ($doNotDelete -eq $duplicateRom)
							{
								$romDND = $true
								break
							}
						}
					}
					
					If ($romDND -eq $true)
					{
						Write-Out -ForegroundColor $global:colorSub1 -Text "DO NOT DELETE (Special Case): $($duplicateRom)" -LogFile $LogFile
					}
					Else 
					{
						If ($primaryRom -eq "")
						{
							#No Primary Found Delete Duplicate
							Write-Out -ForegroundColor DarkRed -Text "DELETE(No Primary Found): $($duplicateRom)" -LogFile $LogFile
						}
						Else
						{
							#Not The Primary ROM Delete Duplicate Or Exception
							Write-Out -ForegroundColor DarkRed -Text "DELETE(Duplicate/Exception): $($duplicateRom)" -LogFile $LogFile
						}					
						#Delete ROM
						For ($i=0; $i -lt $colRoms.Count; $i++)
						{
							If ($colRoms[$i].Name -eq $duplicateRom)
							{
								$colRoms.RemoveAt($i)
								$i--
							}
						}
						$countDuplicatesDeleted++
					}
				}			
			}
		}
		Else 
		{
			#NOTE - $colDuplicateRoms acts as a variable (type=object) because there's a single item in the array!
			#Non-Duplicate ROM Primary By Default
			If ($LogVerbose -eq $true)
			{
				Write-Out -ForegroundColor $global:colorSub1 -Text "SKIP REGEX (No Duplicates): $($sortRegex)" -LogFile $LogFile
				Write-Out -ForegroundColor $global:colorSub1 -Text "Primary ROM Set (Non-Duplicate): $($colDuplicateRoms)" -LogFile $LogFile
			}
			#Check Primary ROM For Exceptions
			$matchRomException = $false
			ForEach ($romException in $romsExceptions) 
			{
				#Check If Current ROM Matches Exception
				If ($colDuplicateRoms -match [regex]::escape($romException))
				{
					#Exception Found (Flag Delete)
					$matchRomException = $true
					#Special Case Do Not Delete ROMs Check
					$romDND = $false
					If ($romsDoNotDelete.Count -gt 0)
					{
						ForEach ($doNotDelete in $romsDoNotDelete)
						{
							If ($doNotDelete -eq $colDuplicateRoms)
							{
								#Found Special Case (Flag NOT Delete)
								$romDND = $true
								break
							}
						}
					}
					break
				}
			}
			#Check to Delete or Not 
			If ($matchRomException -eq $false)
			{
				Write-Out -ForegroundColor $global:colorSub1 -Text "KEEP(Primary Non-Duplicate): $($colDuplicateRoms)" -LogFile $LogFile
			}
			Else 
			{
				#Delete
				If ($romDND -eq $true)
				{
					#Special Case Keep
					Write-Out -ForegroundColor $global:colorSub1 -Text "DO NOT DELETE (Special Case Non-Duplicate): $($colDuplicateRoms)" -LogFile $LogFile
				}
				Else 
				{
					#Delete ROM 
					Write-Out -ForegroundColor DarkRed -Text "DELETE (Exception Non-Duplicate) '$($romException)': $($colDuplicateRoms)" -LogFile $LogFile
					For ($i=0; $i -lt $colRoms.Count; $i++)
					{
						If ($colRoms[$i].Name -eq $duplicateRom)
						{
							$colRoms.RemoveAt($i)
							$i--
						}
					}
				}
			}
		}
	}
	
	
	#Copy NoIntro ROMs
	Write-Out -ForegroundColor $global:colorMain -Text "$($GameSystem.ToUpper()) (STEP 5/6): Copy No-Intro ROMs" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorMain -Text "================================================================" -LogFile $LogFile
	#Confirm Source ROM Directory Exists
	If (Test-Path -LiteralPath $RomSource)
	{
		Write-Out -ForegroundColor $global:colorSub1 -Text "Source Exist: '$($RomSource)'" -LogFile $LogFile
		
		#Confirm Destination Directory Exists
		If (Test-Path -LiteralPath $RomDestination)
		{
			#Destination Directory Exists - Overwrite?
			If ($RomOverwrite -eq $true)
			{
				Write-Out -ForegroundColor $global:colorWarn -Text "Destination EXIST (Overwrite): '$($RomDestination)'" -LogFile $LogFile
				Write-Out -ForegroundColor $global:colorSub1 -Text "Copy $($colRoms.count) ROMs from '$($RomSource)*' to '$($RomDestination)'" -LogFile $LogFile
				ForEach ($rom in $colRoms)
				{
					Copy-Item -LiteralPath $rom.FullName -Destination $RomDestination -Recurse -Force
				}				
			}
			Else 
			{
				Write-Out -ForegroundColor $global:colorWarn -Text "Destination EXIST (Do Not Overwrite): '$($RomDestination)'" -LogFile $LogFile
				Write-Out -ForegroundColor $global:colorWarn -Text "No Files Copied!'" -LogFile $LogFile
			}
		}
		Else 
		{
			Write-Out -ForegroundColor $global:colorSub1 -Text "Destination Not Exist: '$($RomDestination)'" -LogFile $LogFile
			Write-Out -ForegroundColor $global:colorSub1 -Text "Create Folder: '$($RomDestination)'" -LogFile $LogFile
			If (New-Item -ItemType Directory -Path $RomDestination -Force)
			{
				Write-Out -ForegroundColor $global:colorSub1 -Text "Copy $($colRoms.count) ROMs from '$($RomSource)' to '$($RomDestination)'" -LogFile $LogFile
				ForEach ($rom in $colRoms)
				{
					Copy-Item -LiteralPath $rom.FullName -Destination $RomDestination -Recurse -Force
				}
			}
			Else 
			{
				Write-Out -ForegroundColor $global:colorError -Text "ERROR Creating Destination No ROMs Copied!" -LogFile $LogFile
			}
		}
	}
	Else
	{
		Write-Out -ForegroundColor $global:colorError -Text "Source NOT Exist: '$($RomSource)'" -LogFile $LogFile
		Write-Out -ForegroundColor $global:colorError -Text "ERROR CANNOT COPY!!!" -LogFile $LogFile
	}
	#Optionally Unzip ROMs Incase .zip Not Supported By Emulator
	If ($romUnzip -eq $true)
	{
		Write-Out -ForegroundColor $global:colorSub1 -Text "Unzip ROMs In: '$($RomDestination)'" -LogFile $LogFile
		$colRoms = Get-ChildItem -LiteralPath $RomDestination -Filter "*.zip"
		ForEach ($rom in $colRoms)
		{
			Expand-Archive $rom.FullName -DestinationPath $RomDestination -Force
			Remove-Item -LiteralPath $rom.FullName -Force
		}
	}



	#Results
	Write-Out -ForegroundColor $global:colorMain -Text "$($GameSystem.ToUpper()) (STEP 6/6): Processing Results" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorMain -Text "================================================================" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROMs TOTAL              : $($countTotal)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROMs UNIQUE             : $($countUnique)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROMs Exceptions Deleted : $($countExceptionsDeleted)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROMs Dups Deleted       : $($countDuplicatesDeleted)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROMs Non-Dup Deleted    : $($countNonDuplicatesDeleted)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROMs Total Deleted      : $($countExceptionsDeleted + $countDuplicatesDeleted + $countNonDuplicatesDeleted)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "ROMs Total After Clean  : $($countTotal - ($countExceptionsDeleted + $countDuplicatesDeleted + $countNonDuplicatesDeleted))" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "Start Time              : $($timeStart)" -LogFile $LogFile
	$timeEnd = (Get-Date)
	Write-Out -ForegroundColor $global:colorSub1 -Text "End Time                : $($timeEnd)" -LogFile $LogFile
	Write-Out -ForegroundColor $global:colorSub1 -Text "Elapsed Time            : $([math]::Round(($timeEnd-$timeStart).totalseconds, 2)) seconds" -LogFile $LogFile
}



Function Get-NoIntroDefaultFilters
{
    Param (
        [Parameter(Mandatory=$false)]
        [string]
        $Name = "",	
        [Parameter(Mandatory=$false)]
        [string]
        $Unzip = '$false'
    )
    
    Begin
    {
        #Nothing to Do Here
    }
    
    Process
    {

        $DefaultFilters = @"
#$Name v5.3

#Unzip ROMs That Do Not Support .ZIP Files On RetroPie
`$romUnzip = $Unzip
`n
"@        
        $DefaultFilters += @'
#Any Exact ROM Name (Including Extension) In This List Will Not Be Deleted
#-ROMs listed here will NOT be deleted even if they match a value in romsExceptions
#-ROMs listed here will NOT be deleted if it's NOT the Primary ROM via romsSortOrder
#-Ex: "Castlevania (USA) (Rev A).zip"
$romsDoNotDelete = @(
)

#Any ROMs Containing These Strings Or Exact ROM Name Will Be Removed Unless the Exact ROM Name Is In romsDoNotDelete
$romsExceptions = @(
    "[BIOS]"
    ,"[b]"
    ,"(Alpha)"
    ,"(Beta)"
    ,"(Beta 1)"
    ,"(Beta 2)"
    ,"(Beta 3)"
    ,"(Proto)"
    ,"(Proto 1)"
    ,"(Proto 2)"
    ,"(Promo)"
    ,"(Sample)"
    ,"(Debug Version)"
    ,"(Enhancement Chip)"
    ,"(GameCube Edition)"
    ,"(Promo, Virtual Console)"
    ,"(Virtual Console)"
    ,"(Wii Virtual Console)"
    ,"(Wii U Virtual Console)"
    ,"(Asia)"
    ,"(Australia)"
    ,"(Brazil)"
    ,"(Canada)"
    ,"(China)"
    ,"(France)"
    ,"(Germany)"
    ,"(Hong Kong)"
    ,"(Italy)"
    ,"(Japan)"
    ,"(Korea)"
    ,"(Netherlands)"
    ,"(Russia)"
    ,"(Spain)"
    ,"(Sweden)"
    ,"(Taiwan)"
)

#Preferred Sorting Order to Pick 1 Primary ROM to Keep When Evaluating Duplicates
#-Order Selects Specific Names followed by more generalized names based around language
#-ROMs in romsExceptions will not be deleted even if it is not determined as the primary
#-Ex: "Joust (USA).zip" would be primary over "Joust (Japan).zip"
$romsSortOrder = @(
    #---------------------------------------
    #---USA PREFERRED ORDER---
    #---------------------------------------
    #(USA).zip
    "\(USA\)\.zip"
    #(USA) (Unl).zip
    ,"\(USA\)\s*\(Unl\)\.zip"
    #(USA) (En,*).zip
    ,"\(USA\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(USA) (En,*) (Unl).zip
    ,"\(USA\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(USA) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(USA\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"
    #(USA, Europe).zip
    ,"\(USA,\s*Europe\)\.zip"
    #(USA, Europe) (Unl).zip
    ,"\(USA,\s*Europe\)\s*\(Unl\)\.zip"
    #(USA, Europe) (En,*).zip
    ,"\(USA,\s*Europe\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(USA, Europe) (En,*) (Unl).zip
    ,"\(USA,\s*Europe\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(USA, Europe) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(USA,\s*Europe\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"
    #(Japan, USA).zip
    ,"\(Japan,\s*USA\)\.zip"
    #(Japan, USA) (Unl).zip
    ,"\(Japan,\s*USA\)\s*\(Unl\)\.zip"
    #(Japan, USA) (En,*).zip
    ,"\(Japan,\s*USA\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(Japan, USA) (En,*) (Unl).zip
    ,"\(Japan,\s*USA\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(Japan, USA) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(Japan,\s*USA\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"
    #(USA, Asia).zip
    ,"\(USA,\s*Asia\)\.zip"
    #(USA, Asia) (Unl).zip
    ,"\(USA,\s*Asia\)\s*\(Unl\)\.zip"
    #(USA, Asia) (En,*).zip
    ,"\(USA,\s*Asia\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(USA, Asia) (En,*) (Unl).zip
    ,"\(USA,\s*Asia\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(USA, Asia) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(USA,\s*Asia\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"    
    #(USA, Australia).zip
    ,"\(USA,\s*Australia\)\.zip"
    #(USA, Australia) (Unl).zip
    ,"\(USA,\s*Australia\)\s*\(Unl\)\.zip"
    #(USA, Australia) (En,*).zip
    ,"\(USA,\s*Australia\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(USA, Australia) (En,*) (Unl).zip
    ,"\(USA,\s*Australia\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(USA, Australia) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(USA,\s*Australia\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"
    #(USA, Korea).zip
    ,"\(USA,\s*Korea\)\.zip"
    #(USA, Korea) (Unl).zip
    ,"\(USA,\s*Korea\)\s*\(Unl\)\.zip"
    #(USA, Korea) (En,*).zip
    ,"\(USA,\s*Korea\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(USA, Korea) (En,*) (Unl).zip
    ,"\(USA,\s*Korea\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(USA, Korea) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(USA,\s*Korea\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"
    #(USA) (Rev #)
    ,"^(?=.*?\(USA\))(?=.*?\(Rev\s*[0-9\.]{0,4}\)).*$"
    #(USA) (Rev A)
    ,"^(?=.*?\(USA\))(?=.*?\(Rev\s*[A-Z]{0,4}\)).*$"
    #(USA) (Rev #A)
    ,"^(?=.*?\(USA\))(?=.*?\(Rev\s*[A-Z0-9\.]{0,4}\)).*$"
    #(USA) (Alt) OR (Alternate)
    ,"^(?=.*?\(USA\))(?=.*?(\(Alt\)|\(Alternate\))).*$"
    #(USA) (Alt #)
    ,"^(?=.*?\(USA\))(?=.*?\(Alt\s*[A-Z0-9]{1,2}\)).*$"
    #(USA)
    ,"\(USA\)"
    #(USA, Europe)
    ,"\(USA,\s*Europe\)"
    #(Japan, USA)
    ,"\(Japan,\s*USA\)"
    #(USA, Asia)
    ,"\(USA,\s*Asia\)"
    #(USA, Australia)
    ,"\(USA,\s*Australia\)"
    #(USA, Korea)
    ,"\(USA,\s*Korea\)"    
    #(*USA*)
    ,"\((.*?)USA(.*?)\)"
    #---------------------------------------
    #---WORLD PREFERRED ORDER---
    #---------------------------------------
    #(World).zip
    "\(World\)\.zip"
    #(World) (Unl).zip
    ,"\(World\)\s*\(Unl\)\.zip"
    #(World) (En,*).zip
    ,"\(World\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(World) (En,*) (Unl).zip
    ,"\(World\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(World) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(World\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"
    #(World) (Rev #)
    ,"^(?=.*?\(World\))(?=.*?\(Rev\s*[0-9\.]{0,4}\)).*$"
    #(World) (Rev A)
    ,"^(?=.*?\(World\))(?=.*?\(Rev\s*[A-Z]{0,4}\)).*$"
    #(World) (Rev #A)
    ,"^(?=.*?\(World\))(?=.*?\(Rev\s*[A-Z0-9\.]{0,4}\)).*$"
    #(World) (Alt) OR (Alternate)
    ,"^(?=.*?\(World\))(?=.*?(\(Alt\)|\(Alternate\))).*$"
    #(World) (Alt #)
    ,"^(?=.*?\(World\))(?=.*?\(Alt\s*[A-Z0-9]{1,2}\)).*$"
    #(World)
    ,"\(World\)"
    #(*World*)
    ,"\((.*?)World(.*?)\)"
    #---------------------------------------
    #---EUROPE PREFERRED ORDER---
    #---------------------------------------
    #(Europe).zip
    "\(Europe\)\.zip"
    #(Europe) (Unl).zip
    ,"\(Europe\)\s*\(Unl\)\.zip"
    #(Europe) (En,*).zip
    ,"\(Europe\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(Europe) (En,*) (Unl).zip
    ,"\(Europe\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(Europe) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(Europe\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"
    #(Japan, Europe).zip
    ,"\(Japan,\s*Europe\)\.zip"
    #(Japan, Europe) (Unl).zip
    ,"\(Japan,\s*Europe\)\s*\(Unl\)\.zip"
    #(Japan, Europe) (En,*).zip
    ,"\(Japan,\s*Europe\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(Japan, Europe) (En,*) (Unl).zip
    ,"\(Japan,\s*Europe\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(Japan, Europe) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(Japan,\s*Europe\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"
    #(Europe) (Rev #)
    ,"^(?=.*?\(Europe\))(?=.*?\(Rev\s*[0-9\.]{0,4}\)).*$"
    #(Europe) (Rev A)
    ,"^(?=.*?\(Europe\))(?=.*?\(Rev\s*[A-Z]{0,4}\)).*$"
    #(Europe) (Rev #A)
    ,"^(?=.*?\(Europe\))(?=.*?\(Rev\s*[A-Z0-9\.]{0,4}\)).*$"
    #(Europe) (Alt) OR (Alternate)
    ,"^(?=.*?\(Europe\))(?=.*?(\(Alt\)|\(Alternate\))).*$"
    #(Europe) (Alt #)
    ,"^(?=.*?\(Europe\))(?=.*?\(Alt\s*[A-Z0-9]{1,2}\)).*$"
    #(Europe)
    ,"\(Europe\)"
    #(*Europe*)
    ,"\((.*?)Europe(.*?)\)"
    #---------------------------------------
    #---JAPAN PREFERRED ORDER---
    #---------------------------------------
    #(Japan).zip
    "\(Japan\)\.zip"
    #(Japan) (Unl).zip
    ,"\(Japan\)\s*\(Unl\)\.zip"
    #(Japan) (En,*).zip
    ,"\(Japan\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(Japan) (En,*) (Unl).zip
    ,"\(Japan\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(Japan) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(Japan\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"
    #(Japan, Korea).zip
    ,"\(Japan,\s*Korea\)\.zip"
    #(Japan, Korea) (Unl).zip
    ,"\(Japan,\s*Korea\)\s*\(Unl\)\.zip"
    #(Japan, Korea) (En,*).zip
    ,"\(Japan,\s*Korea\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\.zip"
    #(Japan, Korea) (En,*) (Unl).zip
    ,"\(Japan,\s*Korea\)\s*\(En(,|\+|Ca|Da|De|En|Es|Fi|Fr|It|Ja|Nl|No|Pl|Pt|Sv|Zh)*?\)\s*\(Unl\)\.zip"
    #(Japan, Korea) <Anything Except (Alt ) or (Rev ) or (Demo)>.zip
    ,"^(?=.*?\(Japan,\s*Korea\))((?!\(Alt\s*[A-Z0-9]{0,2}\)|\(Rev\s*[A-Z0-9\.]{0,4}\)|\(Demo\)).)*$"
    #(Japan) (Rev #)
    ,"^(?=.*?\(Japan\))(?=.*?\(Rev\s*[0-9\.]{0,4}\)).*$"
    #(Japan) (Rev A)
    ,"^(?=.*?\(Japan\))(?=.*?\(Rev\s*[A-Z]{0,4}\)).*$"
    #(Japan) (Rev #A)
    ,"^(?=.*?\(Japan\))(?=.*?\(Rev\s*[A-Z0-9\.]{0,4}\)).*$"
    #(Japan) (Alt) OR (Alternate)
    ,"^(?=.*?\(Japan\))(?=.*?(\(Alt\)|\(Alternate\))).*$"
    #(Japan) (Alt #)
    ,"^(?=.*?\(Japan\))(?=.*?\(Alt\s*[A-Z0-9]{1,2}\)).*$"
    #(Japan)
    ,"\(Japan\)"
    #(*Japan*)
    ,"\((.*?)Japan(.*?)\)"
    #---------------------------------------
    #---ALL ELSE FAILED PREFERRED ORDER---
    #---------------------------------------
    #(Unknown)
    ,"\(Unknown\)"
)
'@

        Return $DefaultFilters
    }
    
    End
    {
        #Nothing to Do Here
    }
}



Function Write-Out
{
	<#
	.SYNOPSIS
	Write output to console and/or log file. Supports multiple foreground and background colors on a single line and 
	advanced options to insert lines, spaces, or tabs before or after the text as well as timestamps.
	.DESCRIPTION
	Uses Write-Host to output color text to the console, and supports using multiple foreground and background colors 
	on a single line. Uses Set-Content and Add-Content to output to a file.
	
	It works by accepting strings as an array and then you can assign the foreground or background colors by passing
	them as arrays for the matching text. It also accepts Default which allows you skip specifying a color and use the 
	default.
	
	This function requires Powershell 4.0 to support the alias attribute in the function.
	.PARAMETER Text
	Optional. Array of strings to write to console/log. If you do not need to use multiple colors on a single line you can pass a single string instead of an array of strings. Otherwise, if you need to multiple colors on a single line pass an array of strings and associated colors using -ForegroundColor, -BackgroundColor, -ForegroundColorDefault, or -BackgroundColorDefault.
	.PARAMETER ForegroundColor
	Optional. Array of foreground Colors to write to console. Default means the ForegroundColorDefault value is used.
	Valid Colors: Default, Black, DarkBlue, DarkGreen, DarkCyan, DarkRed, DarkMagenta, DarkYellow, Gray, DarkGray, Blue, Green, Cyan, Red, Magenta, Yellow, White
	.PARAMETER ForegroundColorDefault
	Optional. Default foreground color to write to console. Default is set to white.
	Valid Colors: Black, DarkBlue, DarkGreen, DarkCyan, DarkRed, DarkMagenta, DarkYellow, Gray, DarkGray, Blue, Green, Cyan, Red, Magenta, Yellow, White
	.PARAMETER BackgroundColor
	Optional. Array of background colors to write to console. Default means the BackgroundColorDefault value is used.
	Valid Colors: Default, Black, DarkBlue, DarkGreen, DarkCyan, DarkRed, DarkMagenta, DarkYellow, Gray, DarkGray, Blue, Green, Cyan, Red, Magenta, Yellow, White
	.PARAMETER BackgroundColorDefault
	Optional. Default background color to write to console. Default is none.
	Valid Colors: Default, Black, DarkBlue, DarkGreen, DarkCyan, DarkRed, DarkMagenta, DarkYellow, Gray, DarkGray, Blue, Green, Cyan, Red, Magenta, Yellow, White
	.PARAMETER PreLine
	Optional. Add blank lines before your text. Default is 0.
	.PARAMETER PreSpace
	Optional. Add spaces before your text. Default is 0.
	.PARAMETER PreTab
	Optional. Add Tabs before your text. Default is 0.
	.PARAMETER TimeStampFormat
	Optional. Format used for time stamps. Default is 'yyyy-MM-dd HH:mm:ss'
	.PARAMETER TimeStamp
	Optional. Adds time stamp in square brackets before your text.
	.PARAMETER ClearHost
	Optional. Clear the console before your text.
	.PARAMETER PostLine
	Optional. Add blank lines after your text. Default is 0.
	.PARAMETER PostSpace
	Optional. Add spaces after your text. Default is 0.
	.PARAMETER PostTab
	Optional. Add tabs after your text. Default is 0.
	.PARAMETER NoNewLine
	Optional. Do not add a new line after your text and optional post text options. Default is to add a new line.
	.PARAMETER LogFile
	Optional. Absolute or relative path of the log file.
	.PARAMETER Overwrite
	Optional. Ovewrite the log file. Default is to append to the log file.
	.INPUTS
	Parameters above
	.OUTPUTS
	None
	.NOTES
	Author: Brian Steinmeyer
	URL: http://sigkillit.com/
	Created: 4/18/2020
	Version 1.3
	- Added LogOnly Option for only writing text to a log file
	Version 1.2
	- Set Text, ForegroundColor, and BackgroundColor to default value of @() to fix errors checking counts in some circumstances
	- Fixed an issue where ForegroundColorDefault and BackgroundColorDefault were not working properly in some circumstances
	- Added Requires -Version 4.0
	Version 1.1
	- Completely rewrote the "Main Text" section
	- Added "Default" as a color option, which allows you to use the default values for foreground/background 
	  - Useful when you want to specify a backgroundcolor in certain parts of a line like the middle
	  - Ex: Write-Out -Text "How to ","highlight ","the middle text" -BackgroundColor Default,Yellow,Default
	Version: 1.0
	- Initial Creation inspired by PSWriteColor (https://github.com/EvotecIT/PSWriteColor)
	  - Improved upon by switching Foreground and Background Colors to default values if colors are not specifid for all strings.
	    Will also ignore extra colors if more colors are specified than strings specified.
	.EXAMPLE
	Write-Out -Text "Start with Red Text ","Then Switch to Blue Text ","Now Magenta" -ForegroundColor Red,Blue,Magenta 
	.EXAMPLE
	Write-Out -Text "White on Black ","Black on White ","Dark Cyan on Cyan ","Yellow on Green ","Default Color" -ForegroundColor White,Black,DarkCyan,Yellow -BackgroundColor Black,White,Cyan,Green
	.EXAMPLE
	Write-Out -Text "Make this"," entire line"," the same color by setting defaults" -ForegroundColorDefault Yellow -BackgroundColorDefault Magenta
	.EXAMPLE
	Write-Out -Text "Add a blank line and two tabs ","before ","my text" -ForegroundColor Green,Cyan,White -PreLine 1 -PreTab 2
	.EXAMPLE
	Write-Out -Text "Add two blank ","lines ","after my text" -ForegroundColor White,Green,White -PostLine 2
	.EXAMPLE
	Write-Out -Text "Add 3 spaces before my text" -ForegroundColor Gray -Prespace 3
	.EXAMPLE
	Write-Out -Text "White text and a tab after" -ForegroundColor White -NoNewLine -PostTab 1
	Write-Out -Text "Black text on Yellow ","and then back to white" -ForegroundColor Black,White -BackgroundColor Yellow
	.EXAMPLE
	Write-Out -Text "An easy way to ","highlight ","text in the middle" -BackgroundColor Default,Yellow
	.EXAMPLE
	Write-Out -Text "You can even add a ","time stamp ","before your output" -ForegroundColor White,Green,White -TimeStamp -PreLine 3
	.EXAMPLE
	Write-Out -Text "You can change the ","time stamp format" -ForegroundColor White,Yellow -TimeStamp -TimeStampFormat "dd-MM-yyy HH:mm" -PreLine 1 -PostLine 1
	.EXAMPLE
	Write-Out -Text "An"," Error"," occurred let's write overwrite/create a new log file" -ForegroundColor White,Red,White -TimeStamp -LogFile "script.log" -Overwrite
	.EXAMPLE
	Write-Out -Text "Now you can ","Append ","this line to your log file" -ForegroundColor Cyan,Magenta -TimeStamp -LogFile "script.log"
	#>
	[CmdletBinding()]
	Param (
		[Parameter(Mandatory=$false)]
		[AllowEmptyString()]
        [alias ('T')]
		[String[]]
		$Text = @(),
		[Parameter(Mandatory=$false)]
		[ValidateSet("Default","Black","DarkBlue","DarkGreen","DarkCyan","DarkRed","DarkMagenta","DarkYellow","Gray","DarkGray","Blue","Green","Cyan","Red","Magenta","Yellow","White")]
		[alias ('FGC', 'FC')]
		[string[]]
		$ForegroundColor = @(),
		[Parameter(Mandatory=$false)]
		[ValidateSet("Default","Black","DarkBlue","DarkGreen","DarkCyan","DarkRed","DarkMagenta","DarkYellow","Gray","DarkGray","Blue","Green","Cyan","Red","Magenta","Yellow","White")]
		[alias ('FGCD', 'FCD')]
		[string]
		$ForegroundColorDefault = [ConsoleColor]::White,
		[Parameter(Mandatory=$false)]
        [ValidateSet("Default","Black","DarkBlue","DarkGreen","DarkCyan","DarkRed","DarkMagenta","DarkYellow","Gray","DarkGray","Blue","Green","Cyan","Red","Magenta","Yellow","White")]
		[alias ('BGC', 'BC')]
		[String[]]
		$BackgroundColor = @(),
		[Parameter(Mandatory=$false)]
		[ValidateSet("Default","Black","DarkBlue","DarkGreen","DarkCyan","DarkRed","DarkMagenta","DarkYellow","Gray","DarkGray","Blue","Green","Cyan","Red","Magenta","Yellow","White")]
		[alias ('BGCD', 'BCD')]
		[string]
		$BackgroundColorDefault = "Default",
		[Parameter(Mandatory=$false)]
		[int]
		$PreLine = 0,
		[Parameter(Mandatory=$false)]
		[int]
		$PreSpace = 0,
		[Parameter(Mandatory=$false)]
		[int]
		$PreTab = 0,
		[Parameter(Mandatory=$false)]
		[Alias('TSF', 'TS')]
		[string]
		$TimeStampFormat = 'yyyy-MM-dd HH:mm:ss',
		[Parameter(Mandatory=$false)]
		[switch]
		$TimeStamp,		
		[Parameter(Mandatory=$false)]
		[switch]
		$ClearHost,
		[Parameter(Mandatory=$false)]
		[int]
		$PostLine = 0,
		[Parameter(Mandatory=$false)]
		[int]
		$PostSpace = 0,
		[Parameter(Mandatory=$false)]
		[int]
		$PostTab = 0,
		[Parameter(Mandatory=$false)]
		[switch]
		$NoNewLine = $false,
		[Parameter(Mandatory=$false)]
		[alias ('Log', 'L')]
		[string]
		$LogFile = '',
		[Parameter(Mandatory=$false)]
		[switch]
		$LogOnly = $false,
		[Parameter(Mandatory=$false)]
		[switch]
		$Overwrite = $false
	)
	
	Begin
	{
		#Nothing to Do Here
	}
	
	Process
	{
		#Optional - Do Not Write to ConsoleColor
		If (!($LogOnly))
		{
			#Optional - Prefix Text 
			If ($ClearHost) { Clear-Host }
			If ($PreLine -gt 0) { For ($i = 0; $i -lt $PreLine; $i++) { Write-Host -Object "`n" -NoNewline } } # Add empty line(s) before Main text
			If ($PreSpace -gt 0) { For ($i = 0; $i -lt $PreSpace; $i++) { Write-Host -Object " " -NoNewLine } }  # Add Tab(s) before Main text
			If ($PreTab -gt 0) { For ($i = 0; $i -lt $PreTab; $i++) { Write-Host -Object "`t" -NoNewLine } }  # Add Space(s) before Main text
			If ($TimeStamp) { Write-Host -Object "[$([datetime]::Now.ToString($TimeStampFormat))]" -NoNewline } # Add Timestamp before Main Text

			#MAIN TEXT
			If (($Text.Count -gt 0) -AND ($ForegroundColor.Count -eq 0) -AND ($BackgroundColor.Count -eq 0))
			{
				#Text Only Specified
				For ($i = 0; $i -lt $Text.Count; $i++)
				{
					If ($BackgroundColorDefault -eq "Default")
					{
						Write-Host $Text[$i] -ForegroundColor $ForegroundColorDefault -NoNewLine
					}
					Else 
					{
						Write-Host $Text[$i] -ForegroundColor $ForegroundColorDefault -BackgroundColor $BackgroundColorDefault -NoNewLine
					}
				} 
			}
			ElseIf (($Text.Count -gt 0) -AND ($ForegroundColor.Count -gt 0) -AND ($BackgroundColor.Count -eq 0))
			{ 
				#Text and ForegroundColor Specified
				For ($i = 0; $i -lt $Text.Count; $i++)
				{
					If ($ForegroundColor.Count -le $i) { $ForegroundColor += $ForegroundColorDefault } #ForegroundColor Not Specified Set to Default
					If ($ForegroundColor[$i] -eq "Default") { $ForegroundColor[$i] = $ForegroundColorDefault }
					If ($BackgroundColorDefault -eq "Default")
					{
						Write-Host $Text[$i] -ForegroundColor $ForegroundColor[$i] -NoNewLine
					}
					Else 
					{
						Write-Host $Text[$i] -ForegroundColor $ForegroundColor[$i] -BackgroundColor $BackgroundColorDefault -NoNewLine
					}
				}
			}
			ElseIf (($Text.Count -gt 0) -AND ($ForegroundColor.Count -eq 0) -AND ($BackgroundColor.Count -gt 0))
			{
				#Text and BackgroundColor Specified
				For ($i = 0; $i -lt $Text.Count; $i++)
				{
					If ($BackgroundColor.Count -le $i) { $BackgroundColor += $BackgroundColorDefault } #BackgroundColor Not Specified Set to Default
					If ($BackgroundColor[$i] -eq "Default")
					{
						If ($BackgroundColorDefault -eq "Default")
						{
							Write-Host $Text[$i] -ForegroundColor $ForegroundColorDefault -NoNewLine
						}
						Else
						{
							Write-Host $Text[$i] -ForegroundColor $ForegroundColorDefault -BackgroundColor $BackgroundColorDefault -NoNewLine
						}
					}
					Else 
					{
						Write-Host $Text[$i] -ForegroundColor $ForegroundColorDefault -BackgroundColor $BackgroundColor[$i] -NoNewLine
					}
				}
			}
			ElseIf (($Text.Count -gt 0) -AND ($ForegroundColor.Count -gt 0) -AND ($BackgroundColor.Count -gt 0))
			{ 
				#Text, ForegroundColor, and BackgroundColor Specified (FAILS NOT WRITING DEFAULT)
				For ($i = 0; $i -lt $Text.Count; $i++)
				{
					If ($ForegroundColor.Count -le $i) { $ForegroundColor += $ForegroundColorDefault } #ForegroundColor Not Specified Set to Default
					If ($BackgroundColor.Count -le $i) { $BackgroundColor += $BackgroundColorDefault } #BackgroundColor Not Specified Set to Default
					If ($ForegroundColor[$i] -eq "Default") { $ForegroundColor[$i] = $ForegroundColorDefault }
					If ($BackgroundColor[$i] -eq "Default")
					{
						If ($BackgroundColorDefault -eq "Default")
						{
							Write-Host $Text[$i] -ForegroundColor $ForegroundColor[$i] -NoNewLine
						}
						Else
						{
							Write-Host $Text[$i] -ForegroundColor $ForegroundColor[$i] -BackgroundColor $BackgroundColorDefault -NoNewLine
						}					
					}
					Else 
					{
						Write-Host $Text[$i] -ForegroundColor $ForegroundColor[$i] -BackgroundColor $BackgroundColor[$i] -NoNewLine
					}
				}
			}
			Else
			{
				#No Text, ForegroundColor, or BackgroundColor Specified ($Text.Count -eq 0) -AND ($ForegroundColor.Count -eq 0) -AND ($BackgroundColor.Count -eq 0)
			}
				
			#Post Text
			If ($PostSpace -gt 0) { for ($i = 0; $i -lt $PostSpace; $i++) { Write-Host -Object " " -NoNewLine } }  # Add Tab(s) after Main text
			If ($PostTab -gt 0) { for ($i = 0; $i -lt $PostTab; $i++) { Write-Host -Object "`t" -NoNewLine } }  # Add Space(s) after Main text
			If ($PostLine -gt 0) { for ($i = 0; $i -lt $PostLine; $i++) { Write-Host -Object "`n" -NoNewline } } # Add empty line(s) after Main text
			If ($NoNewLine) { Write-Host -NoNewline } else { Write-Host } # Add New Line after Main Text Unless NoNewLine is Supplied
		}
		
		#Log File
		$TextToWrite = "" #Build Text to Write to Log File
		If ($PreLine -gt 0) { For ($i = 0; $i -lt $PreLine; $i++) { $TextToWrite += "`n" } } # Add empty line(s) before Main text
		If ($PreSpace -gt 0) { For ($i = 0; $i -lt $PreSpace; $i++) { $TextToWrite += " " } }  # Add Tab(s) before Main text
		If ($PreTab -gt 0) { For ($i = 0; $i -lt $PreTab; $i++) { $TextToWrite += "`t" } }  # Add Space(s) before Main text
		If ($TimeStamp) { $TextToWrite += "[$([datetime]::Now.ToString($TimeStampFormat))]" } # Add Timestamp before Main Text
		If ($Text.Count -gt 0) { For ($i = 0; $i -lt $Text.Count; $i++) { $TextToWrite += $Text[$i] } } #Add Main Text
		If ($PostSpace -gt 0) { for ($i = 0; $i -lt $PostSpace; $i++) { $TextToWrite += " " } }  # Add Tab(s) after Main text
		If ($PostTab -gt 0) { for ($i = 0; $i -lt $PostTab; $i++) { $TextToWrite += "`t" } }  # Add Space(s) after Main text
		If ($PostLine -gt 0) { for ($i = 0; $i -lt $PostLine; $i++) { $TextToWrite += "`n" } } # Add empty line(s) after Main text		
		If ($LogFile -eq "")
		{
			#No LogFile Specified - Skip
		}
		ElseIf(!(Test-Path -Path $LogFile))
		{
			#Create Log File If Does Not Exist
			If ($NoNewLine)
			{
				$nf = New-Item -Path $LogFile -type file -force -Value $TextToWrite
			}
			Else 
			{
				$nf = New-Item -Path $LogFile -type file -force -Value $TextToWrite`r`n
			}
		}
		Else
		{
			#Log File Exists			
			If($Overwrite)
			{
				#Overwrite Log File
				If ($NoNewLine)
				{
					Set-Content -Path $LogFile -Value $TextToWrite -NoNewline
				}
				Else 
				{
					Set-Content -Path $LogFile -Value $TextToWrite
				}
			}
			Else
			{
				#Append Log File
				Try {
					If ($NoNewLine)
					{
						Add-Content -Path $LogFile -Value $TextToWrite -ErrorAction Stop -NoNewline
					}
					Else 
					{
						Add-Content -Path $LogFile -Value $TextToWrite -ErrorAction Stop
					}
				} Catch {
					Start-Sleep -s 3
					If ($NoNewLine)
					{
						Add-Content -Path $LogFile -Value $TextToWrite -NoNewline
					}
					Else 
					{
						Add-Content -Path $LogFile -Value $TextToWrite
					}
				}
			}
		}
	}
	
	End
	{
		#Nothing to Do Here
	}
}



Main
Aug 31

Remove RDS CALs from RDS Server

Background

There are many circumstances where you will need to remove a RDS CALs from an RDS Server, or in some cases you want to rebuild the entire RD licensing database.  Microsoft allows you to remove an individual CAL license pack using powershell, or rebuild the entire database.  However, if neither of those work, it’s quite easy to manually rebuild the RD licensing database.  I’ve included directions for all 3 methods below, and have tested this on Windows Server 2008, 2008R2, 2012, 2012R2, and 2016.

Remove An Individual RDS CAL License Pack Using Powershell (User or Device CAL)

  • Open powershell elevated as an administrator
  • Type the following command to list the RDS Licenses and note the KeyPackID
    • Alternatively, open RD Licensing Manager and note the Keypack ID
Get-WmiObject Win32_TSLicenseKeyPack
  • Run the below command to remove the licenses pack from your RD Server
    • Replace KEYPACKID with the number you obtained above
wmic /namespace:\\root\CIMV2 PATH Win32_TSLicenseKeyPack CALL UninstallLicenseKeyPackWithId KEYPACKID

 

Rebuild the RD Licensing Database

Microsoft provides directions on how to do this automatically, via a web browser, or via the phone:

https://technet.microsoft.com/en-us/library/dd851428.aspx

 

Manually Rebuild the Licensing Database (Guaranteed to Work if the Previous 2 methods Fail)

  • Make sure you have documentation of your MS License agreement that includes Authorization number, License number, License type (User/Device CAL), and Quantity before proceeding
  • Stop the Remote Desktop Licensing service

Stop Remote Desktop Licensing Service

  • Rename C:\Windows\System32\lserver\TLSLic.edb to C:\Windows\System32\lserver\TLSLic.old
  • Start the Remote Desktop Licensing service
  • All licenses will now be cleared out of RD Licensing Manager, and you’ll need to re-install the licenses you want to add back in
Jul 14

Complex Password Generator

Function Get-RandomPassword {
     param(
         $length = 10,
         $characters = 'abcdefghkmnprstuvwxyzABCDEFGHKLMNPRSTUVWXYZ123456789!"??$%&/()=?*+#_'
     )
     $random = 1..$length | ForEach-Object { Get-Random -Maximum $characters.length }
     $private:ofs=""
     [String]$characters[$random]
 }
 Function Randomize-Text {
     param(
         $text
     )
     $number = $text.length -1
     $indexes = Get-Random -InputObject (0..$number) -Count $number
     $private:ofs=''
     [String]$text[$indexes]
 }
 Function Get-ComplexPassword {
     $password = Get-RandomPassword -length 8 -characters 'abcdefghiklmnprstuvwxyz'
     $password += Get-RandomPassword -length 2 -characters '#*+)'
     $password += Get-RandomPassword -length 4 -characters '123456789'
     $password += Get-RandomPassword -length 6 -characters 'ABCDEFGHKLMNPRSTUVWXYZ'
     Randomize-Text $password
 }
 Get-ComplexPassword
Jul 14

365 Password Generator

This powershell script bulk generates passwords in a similar style as the password generator in Office 365.  The passwords begin with a capital letter, followed by 5 lower case letters, and 2 digits at the end.  You can modify the pattern to suite your needs (Note: It’s using the ASCII table ranges as the set it randomly chooses from).

Function Generate365PW
{
	Param($max = 1)
	For ($i = 0; $i -lt $max; $i++){
		$pw = Get-Random -Count 1 -InputObject ((65..72)+(74..75)+(77..78)+(80..90)) | % -begin {$UC=$null} -process {$UC += [char]$_} -end {$UC}
		$pw += Get-Random -Count 5 -InputObject (97..122) | % -begin {$LC=$null} -process {$LC += [char]$_} -end {$LC}
		$pw += Get-Random -Count 2 -InputObject (48..57) | % -begin {$NB=$null} -process {$NB += [char]$_} -end {$NB}
		write-output $pw
	}
}
Generate365PW 10
Mar 03

Determine If Distribution Group is Being Used in 365 Exchange

“What distribution groups are in use?” and “How many emails are sent to a specific distribution group per month?” are common questions I receive with 365 Exchange or Exchange.  Unfortunately, there is nothing built in that tracks how many emails on sent to a distribution group.  However we can use Get-MessageTrace to count the number of messages sent to a distribution group for a time range with the max being 30 days.  Also note, the by default PageSize returns 1000 items but you can increase the PageSize to 5000 items.  For example, to get the number of emails sent to the distribution group everyone@domain.com for a single day we can use:

$DGCount = Get-MessageTrace -PageSize 5000 -RecipientAddress "everyone@domain.com" -StartDate ([DateTime]::Today.AddDays(-1)) -EndDate ([DateTime]::Today) | ForEach-Object {$count++}
$DGCount

Using this method, we can count the number of emails sent to each distribution group each day and store the results in an output file.  We can then query those output files and create a report.  In my example, the report will show the total emails sent to each distribution group by month and go back 12 months.  Now without further ado, let’s get to the two scripts needed.

365_DGCounter.ps1

#365_DGCounter.ps1
# ------ SCRIPT CONFIGURATION ------
#Log File
$LogFile = $MyInvocation.MyCommand.Path.Replace($MyInvocation.MyCommand.Name,"Logs\") + ((Get-Date).AddDays(-1).ToString('yyyy_MM_dd')) + ".xml"
#Table Name
$TableName = "DG_Emails_Received"
# ------ END CONFIGURATION ------

#Confirm Logs Directory Exists
$LogPath = $MyInvocation.MyCommand.Path.Replace($MyInvocation.MyCommand.Name,"Logs\")
If (!(Test-Path $LogPath)){
	New-Item -ItemType Directory -Force -Path $LogPath
}

#Import Modules and Connect to Office 365
import-module ActiveDirectory
Try {
	$UserCredential = Get-Credential
	$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.outlook.com/powershell/ -Credential $UserCredential -Authentication Basic -AllowRedirection
	Import-PSSession $Session
}
Catch {
	#Error Connecting to Office365
}

#Create Data Table
$CounterTable = New-Object system.Data.DataTable $TableName
$CounterCol1 = New-Object system.Data.DataColumn Date,([datetime])
$CounterCol2 = New-Object system.Data.DataColumn Email,([string])
$CounterCol3 = New-Object system.Data.DataColumn Count,([int])
$CounterTable.columns.add($CounterCol1)
$CounterTable.columns.add($CounterCol2)
$CounterTable.columns.add($CounterCol3)

#Get DG's and Add to Table
$DG = Get-DistributionGroup -Resultsize Unlimited | select Displayname, Primarysmtpaddress
$DG | ForEach-Object {
	$count = 0
	#Note: [DateTime]::Today returns 12:00 AM of the Current Date 
	#      Let's assume the date is currently 3/3/2016, the below returns the range 3/2/2016 12:00 AM to 3/3/2016 12:00 AM
	Get-MessageTrace -PageSize 5000 -RecipientAddress $_.Primarysmtpaddress -StartDate ([DateTime]::Today.AddDays(-1)) -EndDate ([DateTime]::Today) | ForEach-Object {$count++}
	$CounterRow = $CounterTable.NewRow()
	$CounterRow.Date = [datetime](Get-Date).AddDays(-1)
	$CounterRow.Email = $_.Primarysmtpaddress
	$CounterRow.Count = $count
	$CounterTable.Rows.Add($CounterRow)
}

#Output Results
$CounterTable.WriteXml($LogFile)
$CounterTable.WriteXmlSchema($LogFile.replace(".xml",".xsd"))
$CounterTable | format-table -AutoSize

#Disconnect From Office365
Remove-PSSession $Session

Note: [DateTime]::Today returns 12:00 AM of the Current Date.  Let’s assume the date is currently 3/3/2016.  The script above would return the range 3/2/2016 12:00 AM to 3/3/2016 12:00 AM.  Running this script will actually return the email counts from yesterday since today has not ended.

365_DGCounterReport.ps1

#365_DGCounterReport.ps1
# ------ SCRIPT CONFIGURATION ------
#Log File Path
$LogPath = $MyInvocation.MyCommand.Path.Replace($MyInvocation.MyCommand.Name,"Logs\")
#Table Names
$MasterTableName = "DG_Emails_Received"
$ResultTableName = "DG_Emails_Received"
# ------ END CONFIGURATION ------

#Import XML Files to Master Table
$MasterTable = New-Object system.Data.DataTable $MasterTableName
$MasterCol1 = New-Object system.Data.DataColumn Date,([datetime])
$MasterCol2 = New-Object system.Data.DataColumn Email,([string])
$MasterCol3 = New-Object system.Data.DataColumn Count,([int])
$MasterTable.columns.add($MasterCol1)
$MasterTable.columns.add($MasterCol2)
$MasterTable.columns.add($MasterCol3)
Get-ChildItem $LogPath -Filter "*.xml" | Sort-Object Name | ForEach-Object {
	$XmlDocument = [XML](Get-Content -Path $_.FullName)
	$XmlDocument.DocumentElement.$MasterTableName | ForEach-Object {
		$MasterRow = $MasterTable.NewRow()
		$MasterRow.Date = [datetime]$_.Date
		$MasterRow.Email = $_.Email
		$MasterRow.Count = [int]$_.Count
		$MasterTable.Rows.Add($MasterRow)
	}
}

#Get List of Unique Distribution Groups from Master Table
$DGs = @{}
$MasterTable | ForEach-Object {
	If (!($DGs.ContainsKey($_.Email))){
		$DGs.Add($_.Email,$_.Email)
	}
}
$DGs = $DGs.GetEnumerator() | Sort-Object Name

#Count Emails By Month
$ResultTable = New-Object system.Data.DataTable $ResultTableName
$ResultCol1 = New-Object system.Data.DataColumn Email,([string])
$ResultCol2 = New-Object system.Data.DataColumn Mon,([int])
$ResultCol3 = New-Object system.Data.DataColumn Mon_1,([int])
$ResultCol4 = New-Object system.Data.DataColumn Mon_2,([int])
$ResultCol5 = New-Object system.Data.DataColumn Mon_3,([int])
$ResultCol6 = New-Object system.Data.DataColumn Mon_4,([int])
$ResultCol7 = New-Object system.Data.DataColumn Mon_5,([int])
$ResultCol8 = New-Object system.Data.DataColumn Mon_6,([int])
$ResultCol9 = New-Object system.Data.DataColumn Mon_7,([int])
$ResultCol10 = New-Object system.Data.DataColumn Mon_8,([int])
$ResultCol11 = New-Object system.Data.DataColumn Mon_9,([int])
$ResultCol12 = New-Object system.Data.DataColumn Mon_10,([int])
$ResultCol13 = New-Object system.Data.DataColumn Mon_11,([int])
$ResultCol14 = New-Object system.Data.DataColumn Total,([int])
$ResultTable.columns.add($ResultCol1)
$ResultTable.columns.add($ResultCol2)
$ResultTable.columns.add($ResultCol3)
$ResultTable.columns.add($ResultCol4)
$ResultTable.columns.add($ResultCol5)
$ResultTable.columns.add($ResultCol6)
$ResultTable.columns.add($ResultCol7)
$ResultTable.columns.add($ResultCol8)
$ResultTable.columns.add($ResultCol9)
$ResultTable.columns.add($ResultCol10)
$ResultTable.columns.add($ResultCol11)
$ResultTable.columns.add($ResultCol12)
$ResultTable.columns.add($ResultCol13)
$ResultTable.columns.add($ResultCol14)
$CurDate = Get-Date
$DGs | ForEach-Object{
	$DG = $_.Name
	$ResultRow = $ResultTable.NewRow()
	$ResultRow.Email = $DG
	$ResultRow.Mon = 0
	$ResultRow.Mon_1 = 0
	$ResultRow.Mon_2 = 0
	$ResultRow.Mon_3 = 0
	$ResultRow.Mon_4 = 0
	$ResultRow.Mon_5 = 0
	$ResultRow.Mon_6 = 0
	$ResultRow.Mon_7 = 0
	$ResultRow.Mon_8 = 0
	$ResultRow.Mon_9 = 0
	$ResultRow.Mon_10 = 0
	$ResultRow.Mon_11 = 0
	$ResultRow.Total = 0
	$MasterTable | ForEach-Object {
		If ($DG -eq $_.Email){
			#Current Month
			If ($_.Date.Month -eq $CurDate.Month){
				$ResultRow.Mon += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-1
			If ($_.Date.Month -eq $CurDate.AddMonths(-1).Month){
				$ResultRow.Mon_1 += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-2
			If ($_.Date.Month -eq $CurDate.AddMonths(-2).Month){
				$ResultRow.Mon_2 += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-3
			If ($_.Date.Month -eq $CurDate.AddMonths(-3).Month){
				$ResultRow.Mon_3 += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-4
			If ($_.Date.Month -eq $CurDate.AddMonths(-4).Month){
				$ResultRow.Mon_4 += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-5
			If ($_.Date.Month -eq $CurDate.AddMonths(-5).Month){
				$ResultRow.Mon_5 += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-6
			If ($_.Date.Month -eq $CurDate.AddMonths(-6).Month){
				$ResultRow.Mon_6 += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-7
			If ($_.Date.Month -eq $CurDate.AddMonths(-7).Month){
				$ResultRow.Mon_7 += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-8
			If ($_.Date.Month -eq $CurDate.AddMonths(-8).Month){
				$ResultRow.Mon_8 += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-9
			If ($_.Date.Month -eq $CurDate.AddMonths(-9).Month){
				$ResultRow.Mon_9 += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-10
			If ($_.Date.Month -eq $CurDate.AddMonths(-10).Month){
				$ResultRow.Mon_10 += $_.Count
				$ResultRow.Total += $_.Count
			}
			#Current Month-11
			If ($_.Date.Month -eq $CurDate.AddMonths(-11).Month){
				$ResultRow.Mon_11 += $_.Count
				$ResultRow.Total += $_.Count
			}
		}
	}
	$ResultTable.Rows.Add($ResultRow)
}

#Rewrite Column Names as Months
For ($i=0; $i -lt 12; $i++) {
	If ($i -eq 0){
		$ResultTable.columns["Mon"].ColumnName = "$((Get-Culture).DateTimeFormat.GetAbbreviatedMonthName($CurDate.Month)) $($CurDate.Year)"
	}Else{
		$ResultTable.columns["Mon_$($i)"].ColumnName = "$((Get-Culture).DateTimeFormat.GetAbbreviatedMonthName($CurDate.AddMonths($i * -1).Month)) $($CurDate.AddMonths($i * -1).Year)"
	}
}

#Output Results
$ResultTable | format-table -AutoSize
$ResultTable | Export-CSV Report.csv -notypeinformation
Oct 27

List All DNS Records with Powershell

UPDATED 6/16/2016 Thanks for the comments!

Here’s a nice quick script to list all DNS records in each zone on the DNS server(includes sub-zones):

From the DNS Server

$Zones = @(Get-DnsServerZone)
ForEach ($Zone in $Zones) {
	Write-Host "`n$($Zone.ZoneName)" -ForegroundColor "Green"
	$Zone | Get-DnsServerResourceRecord
}

From a Remote DNS Server

$DNSServer = "servernameOrIp"
$Zones = @(Get-DnsServerZone -ComputerName $DNSServer)
ForEach ($Zone in $Zones) {
	Write-Host "`n$($Zone.ZoneName)" -ForegroundColor "Green"
	$Zone | Get-DnsServerResourceRecord -ComputerName $DNSServer
}

From a Remote DNS Server (Output to Tab Delimited File)

$DNSServer = "servernameOrIp"
$Zones = @(Get-DnsServerZone -ComputerName $DNSServer)
ForEach ($Zone in $Zones) {
	Write-Host "`n$($Zone.ZoneName)" -ForegroundColor "Green"
	$Results = $Zone | Get-DnsServerResourceRecord -ComputerName $DNSServer
	echo $Results > "$($Zone.ZoneName).txt"
}

 

May 14

Search for Emails in a 365 User’s Mailbox

EDIT: Search-Mailbox has been deprecated as of April 2020 in 365.  Please see my updated post that about using Compliance Search instead!

Overview

Often times, my posts are influenced by the questions of others in IT forums.  The other day, an IT pro asked “How can I retrieve emails a 365 user sent to a certain recipient”?  Obviously, I thought to myself, there should be a way to search a mailbox with powershell.  While writing the small script to answer their question, I realized I could do more than just search and copy with the search-mailbox cmdlet.

  • Search recoverable items.  This can be useful if a terminated employee deleted important emails that their manager needs.
  • Delete Emails.  This can useful for a scenario where a virus makes it to all user’s inbox or a disgruntled employee emails a nasty email to everyone.
  • There’s a TON of properties indexed by Exchange that you can query

Without further ado, let’s get to the script

Prerequisites

Delegate Full Access to Mailboxes

In order to search mailboxes, you’ll need to ensure your account has Full Access to each user’s mailbox.  You can do this through the 365 Exchange Admin Center, or you can give yourself full access to all user’s mailbox with the following powershell script.  Make sure you authenticate using an Exchange Admin and replace bsteinmeyer@yourdomain.onmicrosoft.com with the account you need to delegate access.

#Connect to 365 with Admin Credentials
$UserCredential = Get-Credential
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic -AllowRedirection
Import-PSSession $Session

#Delegate Full Access to All Mailboxes
Get-Mailbox -ResultSize unlimited -Filter {(RecipientTypeDetails -eq 'UserMailbox')} | Add-MailboxPermission -User bsteinmeyer@yourdomain.onmicrosoft.com -AccessRights FullAccess -InheritanceType all

Search Mailbox For Email Sent to a Specific Email

#Connect to 365 with Admin Credentials
$UserCredential = Get-Credential
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic -AllowRedirection
Import-PSSession $Session

#Search Mailbox
Get-Mailbox jsmith@yourdomain.onmicrosoft.com | Search-Mailbox -SearchQuery "To:foo@bar.com" -TargetMailbox bsteinmeyer@yourdomain.onmicrosoft.com -TargetFolder "Search_Result" -LogLevel Full –SearchDumpster
  • jsmith@yourdomain.onmicrosoft.com = User’s mailbox you want to search
  • foo@bar.com = Email address sent to
  • bsteinmeyer@yourdomain.onmicrosoft.com = The Mailbox you want to copy the emails to
  • SearchDumpster = Search recoverable items (Emails that were deleted from the Trash)
  • *Note: If you only want to test the command and NOT copy anything, you can add the -LogOnly switch

The above will search a specified user’s mailbox for all emails sent to the specified email address.  The results and emails will be copied to the specified mailbox in the specified folder (This will most likely be your admin account).  If the folder does not exist, it will be automatically created.

Search All Mailboxes for Specific Email and Delete It

In order to delete emails with the -DeleteContent switch, you must be assigned the Discovery Management role and Mailbox Import Export role.  By default, the Mailbox Import Export role isn’t assigned to any role group, so we’ll need to create a new group and assign our user.

#Query Discovery Management Members
Get-RoleGroupMember -Identity "Discovery Management"

#Assign Discovery Management Member
Add-RoleGroupMember -Identity "Discovery Management" -Member bsteinmeyer@yourdomain.onmicrosoft.com

#Create Mailbox Import Export Management Group
New-RoleGroup "Mailbox Import-Export Management" -Roles "Mailbox Import Export"

#Add User to Mailbox Import Export Management Group
Add-RoleGroupMember "Mailbox Import-Export Management" -Member bsteinmeyer@yourdomain.onmicrosoft.com

With that complete, we can now search everyone’s email by the subject and date and delete it.

Get-Mailbox -ResultSize Unlimited | Search-Mailbox -SearchQuery {Subject:"You're a Winner!" AND Sent:"5/14/2015"} -DeleteContent -LogLevel Full –SearchDumpster

*Note: If you only want to test the command and NOT delete anything, you can add the -LogOnly switch

Final Comments

If you’d like to further refine your queries or do more advanced queries, see the complete message properties indexed by Exchange Search below:

https://technet.microsoft.com/en-us/library/jj983804(v=exchg.150).aspx

Apr 28

Determine If A Date Is Between Two Dates

If you need to know if a date is between two dates, you can easily figure this out by treating the date as a number and doing comparisons.  This can be useful for instances where you need a script to do a different task on different months, days, years, etc.  Let’s start with our first example, which will demonstrate how the comparison works.

Example 1 – Time Matters

Function IsBetweenDates2([Datetime]$start,[Datetime]$end)
{
	$d = get-date
	if (($d -ge $start) -and ($d -le $end))
	{
		return $true
	}
	else
	{
		return $false
	}
}
IsBetweenDates "4/1/2015 12:00:00 AM" "4/27/2015 12:00:00 PM"

In this example, if the current date/time is 4/27/2015 12:01:00 PM then it would return False due to the time being outside of our defined end range.  If the date/time was 4/27/2015 12:00:00 PM then it would return True.  If you do not specify the time then it will automatically be defined as 12:00:00 AM in the $start and $end variables; it is important to understand this or else it will affect your date comparisons.  If you do not want time to be a factor, I’d suggest always defining your start time as 12:00:00 AM and your end time as 11:59:59 PM.

Example 2 – Year Does Not Matter

Function IsBetweenDates([Datetime]$start,[Datetime]$end)
{
	$d = get-date
	$s = get-date "$($start.Month)/$($start.Day)/$($d.Year)"
	$e = get-date "$($end.Month)/$($end.Day)/$($d.Year)"
	if (($d -ge $s) -and ($d -le $e))
	{
		return $true
	}
	else
	{
		return $false
	}
}
IsBetweenDates "4/1/2016 12:00:00 AM" "4/30/2016 11:59:59 PM"

In this example, I’m replacing the year from any specified date and making it the current year.  This allows us to focus our comparison based on the month, day, and time.  For example, if the current date/time is 4/27/2015 12:00:00 PM and want to know if our script is running during April.  We can input the start date/time as 4/1/2016 12:00:00 AM our end date/time as 4/30/2016 11:59:59 PM and the script would return True despite the year not matching.

Conclusion

With a basic understanding of compairing dates in powershell, you can easily customize the above scripts to meet any custom needs.

Feb 18

Mail Merge with Attachment

There is not native way to add an attachment when doing an mail merge in Microsoft Office (Outlook/Word/Excel).  However, there are 3rd party apps that allow you to add attachments when doing a mail merge, but these programs usually cost $.  If you’re like me and don’t want to spend money on an application you’ll probably use once, then I came up with an alternative free solution with Powershell.  This method works similar to a Microsoft Office mail merge because it will require a data source, email template, and an Outlook profile.

Step 1 – Create Data Source

I will use a users.csv as the data source of users we want to email.  The users.csv file will look as follows:

FirstName LastName Email Username Password
Joseph smith jsmith@domain.com jsmith VideoGame01
Bill Contoso bcontoso@company.com bcontoso LilyFlower18
Jim Rufus jim.rufus@yahoo.com jrufus StuffedAnimal23

Step 2 – Create Outlook Profile to Send Mail Merge From

Now that you have your data source, you’ll need to make sure you have an Outlook profile.  This profile should be setup with the email address you wish to send the mail merge from.

Step 3 – Create Powershell Script

In the below powershell script, you’ll need to modify the following variables:

$DataSourcePath to the data source (users.csv) file created in step 1
$AttachFile – Path to the file to attach to the email
$EmailSubject – Subject of the email

In addition to modifying the above variables, you’ll need to modify $Mail.Body, which is the body of the email.  The below example is referencing data fields in users.csv for the mail merge, which you may want to modify.  These correspond as follows:

$($_.FIRSTNAME) FIRSTNAME field in users.csv

$($_.LASTNAME) LASTNAME field in users.csv

$($_.USERNAME) USERNAME field in users.csv

$($_.PASSWORD) PASSWORD field in users.csv

Special Note:  To add a new line in the body text use a backtick + n ( `n

$DataSource = "C:\mailmerge\users.csv"
$AttachFile = "C:\mailmerge\Setup Email Directions.docx"
$EmailSubject = "How to Setup Email"

Import-CSV $DataSource | Foreach-Object {
	$ol = New-Object -comObject Outlook.Application  
	$Mail = $ol.CreateItem(0)
	$Mail.Recipients.Add($_.EMAIL)	
	$Mail.Attachments.Add($AttachFile)
	$Mail.Subject = $EmailSubject	
	$Mail.Body = "$($_.FIRSTNAME) $($_.LASTNAME),`n`n"
	$Mail.Body += "The attached directions will guide you through setting up your email account.  Your username and password are as follows:`n`n"
	$Mail.Body += "Username: $($_.USERNAME)"
	$Mail.Body += "Password: $($_.PASSWORD)`n`n"
	$Mail.Body += "If you have any issues installing, please contact support"
	$Mail.Send()
}

Extra Special Note:  If you want to have the email body be HTML formatted instead of Plain Text, just modify $Mail.Body to $Mail.HTMLBody and add your HTML tags in the text.  Using the example above:

$DataSource = "C:\mailmerge\users.csv"
$AttachFile = "C:\mailmerge\Setup Email Directions.docx"
$EmailSubject = "How to Setup Email"

Import-CSV $DataSource | Foreach-Object {
	$ol = New-Object -comObject Outlook.Application  
	$Mail = $ol.CreateItem(0)
	$Mail.Recipients.Add($_.EMAIL)	
	$Mail.Attachments.Add($AttachFile)
	$Mail.Subject = $EmailSubject	
	$Mail.HTMLBody = "<html><body>"
        $Mail.HTMLBody += "<h1>$($_.FIRSTNAME) $($_.LASTNAME),</h1>"
	$Mail.HTMLBody += "<p>The attached directions will guide you through setting up your email account.  Your username and password are as follows:</p>"
	$Mail.HTMLBody += "<p>Username: $($_.USERNAME)</p>"
	$Mail.HTMLBody += "<p>Password: $($_.PASSWORD)</p>"
	$Mail.HTMLBody += "<p>If you have any issues installing, please contact support<p></body></html>"
	$Mail.Send()
}

 

Step 4 – Send the Mail Merge

Open Microsoft Outlook with the profile created in step 2 (It is required for Outlook to be open in order for the powershell script to work!)  Then open Powershell, and run the powershell script in step 3.  You can confirm the emails are sending by looking in the “Sent Items” in Outlook.

Note: – If you’re using User Account Control (UAC), Outlook and Powershell must be running at the same security level.  This simply means, if you open powershell using “Run as administrator” you must open Outlook with “Run as administrator”.  Alternatively, if you open powershell normally (not elevated) you must open Outlook normally (not elevated).