In a recent experiment, JobSage, an employer review site, prompted ChatGPT to write emails addressing fifteen sensitive management scenarios, then had law, human resources and management professionals rate the emails.
A study of business owners had found that 50 per cent had already implemented the chatbot, and JobSage wondered whether it could be a thoughtful manager.
The responses were marked by experts and the key takeaways revealed:
For sensitive management scenarios, 60 per cent of responses were found to be acceptable, while 40 per cent of responses were given a net failure rating.
ChatGPT generally performed better when addressing diversity and performed worse addressing compensation and underperforming employees.
In addition, ChatGPT earned its strongest marks when addressing an employee being investigated for sexual harassment, and a company switching healthcare providers to cut costs.
The weakest performance for ChatGPT was when it was asked to respond to an employee concerned about pay equity, a company that needs people to work harder than ever, and a company’s freeze of raises despite record payout to the chief executive officer.
Here are some examples of the email responses from ChatGPT:
Example: Notifying an employee that they are being investigated for sexual harassment
Experts’ rating – Very positive: ChatGPT delivered a direct email, which well matched the serious tone of the scenario, the study found. The legal expert noted that it would have been outstanding but for the fact that it promises to keep the employee informed as to the progress of the investigation, instead of just the outcome. This highlights a limitation of ChatGPT in that it isn’t aware of how individual companies may handle certain situations, JobSage says.
Subject: Notice of Sexual Harassment Investigation
Dear [Employee Name],
I am writing to inform you that a complaint has been received against you regarding an alleged incident of sexual harassment in the workplace. We take all allegations of sexual harassment very seriously and it is our obligation to investigate any such claims.
An investigation has been initiated and we will be conducting interviews with relevant parties to gather as much information as possible. This process may take some time to complete, but please know that we are committed to ensuring a thorough and impartial investigation.
During the investigation, we ask that you fully cooperate with the investigator, and provide any information or documentation relevant to the matter. We also ask that you do not retaliate against the complainant or any other individuals involved in this matter.
Please be aware that sexual harassment is a serious violation of our company policy and could result in disciplinary action, up to and including termination of employment. We will keep you informed of the progress of the investigation and will advise you of any next steps as the investigation progresses.
If you have any questions, please do not hesitate to contact me. We appreciate your cooperation in this matter.
Sincerely,
[Your Name]
[Your Title]
[Your Contact Information]
Example: Notifying employees that the company is switching healthcare providers to cut costs.
Experts’ rating – Very positive: ChatGPT was able to effectively deliver the news about the change in providers with empathy and information, JobSage said. It was also able to acknowledge that employees may be disappointed by this switch for a number of reasons.
Subject: Change in Insurance Provider
Dear [Company Name] Team,
I am writing to inform you of a change to our health insurance provider. We understand that healthcare costs are increasing and that this is an issue that affects us all. In an effort to keep our benefits costs manageable, we have decided to switch to a new insurance provider.
We know that change can be difficult, and that some of you may be disappointed that we are not staying with our previous provider. We want you to know that we carefully considered all available options and made the decision that we believe is in the best interest of our company and our employees.
While we understand that some of you may have a preference for our previous provider, we believe that our new provider will offer quality coverage at an affordable price. We will be holding informational sessions in the coming weeks to provide more details about the new insurance plan and answer any questions you may have.
We encourage you to review the plan documents carefully and reach out to our benefits team if you have any questions or concerns. We want to reassure you that we are committed to providing competitive and valuable benefits to our employees, and that this change will not compromise the quality of the care you receive.
We appreciate your understanding and support during this time of transition. Please know that we are here to support you and answer any questions you may have as we make this change.
Best regards,
[Your Name]
[Your Title]
[Your Contact Information]
Example: Notifying an employee that they have been terminated because the business is not doing well
Experts’ rating – Positive: While ChatGPT did earn acceptable marks across the board for this email and managed to respond with empathy, the management expert pointed out that this should be an in-person conversation. Of survey respondents who have been laid off, 77 per cent said that layoffs should always be communicated in person. Despite writing an acceptable email, ChatGPT does not replace a manager for this situation.
Subject: Termination of Employment due to Financial Constraints
Dear [Employee Name],
I regret to inform you that due to financial constraints and a decrease in revenue, we must make the difficult decision to terminate your employment with our company effective immediately. This decision was not an easy one, and we understand the impact it may have on you and your family.
We value the contributions you have made to our company and appreciate your hard work and dedication to your role. Unfortunately, the current economic situation has left us with no other option but to make this difficult decision.
We want to assure you that this decision is in no way a reflection of your performance or the quality of your work. It is simply a measure we have to take to ensure the long-term viability of our company.
We will be providing you with the severance package, which includes [specify the details of severance package] as a token of appreciation for your contribution to the company. Additionally, we will provide you with a reference letter to assist you in your future job search.
Please let us know if you have any questions or concerns, and we will do our best to address them. We wish you all the best in your future endeavours.
Sincerely,
[Your Name]
[Your Title]
[Your Contact Information]
Example: Notifying an employee they were being terminated for not working hard enough
Experts’ rating – Negative: JobSage says the experts involved in the experiment had issues with ChatGPT’s response in this situation. It emphasized the employee’s performance as compared to peers and offered an overall negative tone that would potentially make its recipient feel bad.
Subject: Termination of Employment due to Performance Issues
Dear [Employee Name],
I regret to inform you that your employment with our company will be terminated, effective immediately. After careful consideration and evaluation, it has been determined that your work performance has not met the expectations set for your role. Specifically, your work ethic and productivity are not on par with your colleagues, despite repeated feedback and performance coaching.
We understand that this news may come as a surprise to you, and we want to assure you that this decision was not made lightly. Our company has high standards for our employees, and unfortunately, your performance has not met those expectations. As a result, we must take action to ensure that our company remains competitive and successful.
We will be providing you with [specify any severance package or benefits]. Additionally, we will provide you with a reference letter to assist you in your future job search.
We wish you all the best in your future endeavours, and we hope that you will be able to use this experience as a valuable learning opportunity for future professional growth and development.
Sincerely,
[Your Name]
[Your Title]
[Your Contact Information]
Example: Notifying an employee that they’ve worn clothing that’s revealing and inappropriate
Experts’ rating – Negative: According to JobSage, ChatGPT failed to understand how language can be judgmental, and its response was less than informative. Its use of the word “revealing” to describe the clothing is subjective and the HR expert provided the feedback that it “screams sexism and provides no meaningful detail about what the policy is and what part they violated.”
Subject: Professional Dress Code Reminder
Dear [Employee Name],
I am writing to address a concern that has been brought to our attention by some of your colleagues. We have received reports that your clothing choices in the workplace have been seen as too revealing or inappropriate.
As you know, our company has a professional dress code policy, and we ask all employees to dress in a manner that is appropriate for the workplace. Our dress code policy is in place to ensure a respectful and professional environment for all employees and to avoid any potential distractions or discomfort for your colleagues.
We understand that everyone has their own personal style, but we want to remind you that the attire you wear at work should reflect a professional image that is aligned with our company values. We encourage you to review our company’s dress code policy and ensure that your clothing choices are consistent with our expectations.
We want to offer you the opportunity to address this matter, and we encourage you to be aware of the concerns that have been raised by your colleagues. We value you as an employee and we want to support you in maintaining a professional demeanour at all times.
Please let us know if you have any questions or concerns about our dress code policy or your clothing choices, and we will work with you to ensure that you are meeting our expectations.
Sincerely,
[Your Name]
[Your Title]
[Your Contact Information]
