Preheat the oven to 350°F. Line a 12-cup muffin pan with baking cups.
In a medium bowl, whisk together flour, brown sugar, pumpkin pie spice, baking soda and salt.
In a separate medium bowl, whisk together the pumpkin purée, buttermilk, eggs, olive oil and vanilla extract.
Add the wet ingredients to the dry ingredients and stir just until combined.
Stir in the chopped nuts or chocolate chips (optional) then scoop the batter into the cups, filling each 3/4 full.
Bake the muffins until a toothpick inserted comes out clean, about 25 minutes. Remove them from the oven and allow them to cool in the pan for 5 minutes before transferring to a rack to cool completely.