Legacy income worth £2.8bn a year but growth likely to slow due to Brexit effect