His career led him first into industry, more precisely, into power utilities. In 1952, he started out as an executive assistant to management at the headquarters of Ruhrgas AG in Essen. In 1955, he moved on to Vereinigte Elektrizitätswerke Westfahlen AG (VEW) in Dortmund. He played a substantial role there in preparing and carrying through its partial privatisation in 1966. One year later, he became the CFO on the management board of VEW.
 
In January 1970, Herrhausen joined Deutsche Bank as a deputy member of the bank's management board. In 1971 he became a full member of the board. He was responsible for international business (North and South America, Australia/ New Zealand, South Africa) and for foreign trade financing. In addition, he supervised the bank's activities in the region of Essen.

At the end of 1974 he was asked by Germany's finance minister to join the commission on "Fundamental Issues of Finance Industry" (The Committee on Banking Structures). The task of the commission was to examine the structure of the German banking industry and make suggestions for its improvement. In 1983, together with two other top managers, Herrhausen became a "steel moderator"; at the request of the federal government, they developed a concept for restructuring the German steel industry.
 
In May 1985, Alfred Herrhausen was elected chairman of the management board of Deutsche Bank, an office that he held together with Dr. F. Wilhelm Christians until the latter retired in May 1988. The two chairmen were jointly responsible for managing a world-wide bank. On the 11 May 1988, Herrhausen became the sole chairman of Deutsche Bank.
 
On 30 November 1989, Alfred Herrhausen was murdered by terrorists in Bad Homburg v.d.H.