Preheat oven to 140C. Pierce lamb in 16 places with a small sharp knife and press some anchovy, some capers and some rosemary sprigs into each cut. Place in a roasting pan, pour over wine and stock and roast until meat is tender (3½-4 hours), topping up with extra stock if pan begins to dry out.
Meanwhile, place potatoes and garlic in a roasting pan, drizzle with oil, season to taste and roast, turning occasionally, until crisp and golden (1¼-1½ hours).
Meanwhile, for salsa verde, soak bread in vinegar until soft (1-2 minutes). Process anchovies, cornichons, capers, garlic and bread in a food processor to a coarse paste, add remaining ingredients, pulse to combine, season to taste and serve with lamb and potatoes, drizzled with pan juices.