“True” peri-peri as the vast majority of people use it absolutely has lemon, garlic, etc. If you want to make a pepper-infused olive oil, go ahead, but it’s not going to be what anyone really thinks of as peri-peri sauce. To the extent that we know the origins about the sauce, they all seem to include talk of more than just peppers in oil. These arguments over authenticity are not half as important as just going ahead and doing.
Additionally, if you want a spicy olive oil, I feel compelled to point out that while it may be obvious to some, you should only infuse the completely dried peppers / spice.