The Brown Lady is famous mostly as being one of the most reliably photographed ghost in history. Although she has not been seen since 1936, she is said to wear a long brown dress or cape. No one knows who the Brown Lady is, or how she is connected to Raynham Hall.

The first sighting was reported in 1835 by a house guest, Colonel Loftus. He actually viewed her twice. He said she was wearing a brown satin dress and had only black empty sockets for eyes. Another sighting was made by Captain Frederick Marryat. He intentionally slept in the "haunted room," but instead caught a glimpse of the Brown Lady an upstairs hallway. His description was the same as Loftus', except this time the Brown Lady was carrying a lantern. Marryat happened to have a gun with him, and fired point-blank at the figure. The bullets, of course, passed right through the ghost. The ghost was not reported again until 1926, at which time it was viewed by two little boys. In 1936, the famous photograph was taken by photographers Captain Provand and Indre Shira during a shoot for the magazine -Country Life-. Shira saw the ghost on the stairs, an instructed Provand to take a picture.