Prep the potatoes: Preheat oven to 425°F. Toss baby potatoes with 2 tbsp olive oil, salt, pepper, garlic powder, oregano, and smoked paprika on a lined sheet pan. Spread out evenly and roast for 15 minutes.
Season the lamb: While potatoes roast, toss lamb bites with the remaining 1 tbsp olive oil, a pinch of salt, and a squeeze of lemon juice. Let sit while the potatoes get a head start.
Add the lamb to the pan: After 15 minutes, push the potatoes to one side and add the lamb bites to the sheet pan. Roast for 10-12 minutes, stirring halfway through, until the lamb is browned on the outside and tender inside.
Finish & serve: Squeeze extra lemon juice over everything and sprinkle with fresh parsley and feta cheese if using. Serve warm with extra lemon wedges!