There is an ejector that is pinned into the trigger group. It is possible that the extractors are not gripping the case hard enough or that the ejector is broken/missing.
Go to www.gunpartscorp.com
and there should be a diagram for the H-S Model 34. The ejector is part number 51. Compare the picture with your gun and you may be able to tell what is wrong and order the part if necessary.