Replaceメソッドもしくは-replace演算子を使う。-replaceは正規表現を使うことができる。
PS > "Kishida Cabinet".Replace("Kishida", "Abe")
Abe Cabinet
PS > "Kishida Cabinet" -replace "Kishida", "Abe"
Abe Cabinet
PS > "Kishida Cabinet" -replace '(.*) (.*)', '$1 and Abe $2'
Kishida and Abe Cabinet