Preheat the oven to 350°F (175°C). Grease and flour an 8-inch round cake pan.
In a medium bowl, whisk together the flour, baking powder, and salt. Set aside.
In a large bowl, beat the softened butter and granulated sugar together until light and fluffy, about 3 minutes.
Add the eggs one at a time, beating well after each addition.
Mix in the lemon juice and lemon zest.
Alternately add the dry ingredients and milk to the butter mixture, beginning and ending with the dry ingredients. Mix until just combined.
Pour the batter into the prepared cake pan and smooth the top.
Bake for 28 to 30 minutes, or until a toothpick inserted into the center comes out clean.
Allow the cake to cool completely in the pan on a wire rack.
While the cake cools, prepare the whipped cream by beating the heavy cream, powdered sugar, and vanilla extract together in a chilled bowl until stiff peaks form.
Cut the cooled lemon cake into 1-inch cubes.
In a large trifle bowl or individual serving glasses, layer one-third of the cake cubes, followed by one-third of the lemon curd, one-third of the whipped cream, and a few fresh blueberries and raspberries.
Repeat the layers two more times, finishing with whipped cream and a few berries on top for garnish.
Refrigerate the assembled trifle for at least 1 hour before serving to allow the flavors to meld.