The studio release said Townsend Harris was the first western diplomat to enter Japan and somebody asked the director who should play Harris. [John] Huston... "Only one man is right for him and that's John Wayne..."
...When Harris set up his first residence, he hired a young washing woman to maintain his quarters. On her second visit, he noticed that the girl had a skin infection and promptly fired her. From this brief encounter, however, a Japanese legend flowered about the love between a beautiful geisha and her "barbarian" from the West. In reality, Harris was a devout Christian who never touched alcohol and refused to work on Sundays. He did not like Japanese women--perhaps he didn't like women at all, since he died a bachelor at the age of seventy-three--and he never would have condoned such immoral conduct. Legend is more powerful than facts and the severe diplomat became a romantic folk hero at the dawn of Japan's fateful opening to the West.
The authentic Harris and the first resident barbarian of Japanese folklore were much too tame for John's taste--and Wayne's persona. The way Huston saw it--and his friend Charles Grayson began writing it in the screenplay--Harris would indeed become involved with a beautiful geisha but he would also be spied upon and attacked by a combative samurai, become the enemy of the governor of Shimoda and, to combat a cholera epidemic, burn down a village.