On the shrine grounds

 In the corner of the shrine grounds there was a large bush covered with blooming flowers.

I walked around the outside to the back of the shrine and found another entrance to the grounds. Just inside the gate there is a large stele but it is too worn to read the inscription.