Assuming GDPR apply to your organization, yes you should worry about GDPR.
PCI-DSS compliance doesn't imply GDPR compliance.
There are many concept present in GDPR and not in PCI-DSS or not with the same scope/expectations. For me you will have to take a close look of some principle of the GDPR like :
- Admissibility of the user for the processing of his personal data
- Necessity of the data processing
- Transparency to the data subject
- Limitation of use to specific purpose
- Principle of data reduction (collecting as little personal data as possible)
- Data deletion
- Data controller and subcontractor
The two regulations have different scopes.
The GDPR scope is much more wider than PCI-DSS which is focusing on the handling and protection of card holder data. GDPR scope include all the personal data your organization is potentially handling. So it will affect many different department of you organization and not only the part which is manipulating card holder data. For example your HR department will be in scope of GDPR for sure as they are collecting & storing personal data of the company employees.