Unexpected profile changes

We've resolved the incident and restored all data that was inadvertently modified.

The bug was triggered when a profile's activity was accessed, either by an administrator through the website or programmatically via a third-party integration. Profiles that were affected between Friday, October 17 at 12:34 PM PT and Sunday, October 19 at 9:49 AM PT have had their attributes (such as membership type, campus, birthday, and gender) and / or permissions reset to their state from immediately before the incident began. Contact data such as phone numbers and email addresses was unaffected.

What this means:
For affected profiles, any intentional changes made to attributes or permissions during that timeframe were unfortunately lost in the restoration process—with one important exception: if an administrator intentionally removed someone's overall account permissions during this period, those removals were preserved. No one has elevated permissions beyond what they should have.
However, changes to access for specific resources (such as individual Workflows or Lists) may have been restored to their pre-incident state. We recommend reviewing these if you made intentional changes during this window.

What you can do:
Organization Administrators can visit https://accounts.planningcenteronline.com/affected_profiles to see profiles in your account that may have been affected by this issue. We recommend reviewing these profiles to ensure everything is as it should be. Please note that the Activity Tab on affected profiles may contain some confusing entries related to the data restoration process.

Additional details:
* Approximately 3% of churches were impacted by this incident, affecting roughly 0.01% of all profiles
* The median number of affected profiles per church is 1, though a very small number of churches will see significantly more profiles affected—in some cases, hundreds
* This issue was unrelated to but complicated by the recent AWS outage
* We have confirmed that no unauthorized access to data occurred during this incident

Our teams are conducting a thorough post-incident analysis to identify improvements to prevent similar issues and enhance our response processes.

We sincerely apologize for the disruption and appreciate your patience as we worked to resolve this issue. An email with similar information as this update has been sent to Organization Administrators of affected accounts.

The large majority of restore work has been completed. The team continues to clean up some final pieces. We will post another update in the morning.

We've completed the Groups restoration and are continuing to work towards fully restoring People collaborator permissions for things like workfows, lists, etc.

We've completed our data restore work in Home, Services, Chat, Calendar, Registrations, Giving, Publishing, and Check-Ins. We're getting closer to restoring the secondary permission for People and Groups.

Our team is continuing to make progress restoring and validating affected data and permissions. If you are still experiencing issues with logging in, in some cases clearing your browser's cache has shown to be effective.

We've notified affected organization administrators with details about the issue, impact, and our estimated completion timeframe. The team is continuing to make progress restoring permissions and data across our products.

Recovery work has resumed and is progressing well. We've restored core profile data and are validating that restoration, while also working to restore associated data across all applications.

Recovery efforts have been paused due to the major outage in our upstream provider.

We will continue work in the morning. If any profiles in your account were affected you will receive an email with specifics later this week.

We're still actively working on this. Thank you for your patience.

We've identified the users who were impacted. We're now evaluating what changes we can revert and will update as soon as we know more.

We're continuing to investigate to determine our options. We'll share an update as soon as we have more information.

We've implemented a fix to prevent further profile changes. We are now working to understand our options for affected profiles and will update as soon as we have clarity.

Our team is still actively investigating reports of unintended profile updates and permission updates. We’ll share updates as soon as we know more.

We are currently investigating reports of unintended profile updates and permission changes occurring during page refreshes.