Salesforce Monitoring in ELK and an Automated Device Data Feed for a Global Consumer Goods Manufacturer

The challenge

A global consumer goods manufacturer ran its platforms on AWS with an internal DevOps team of 6 developers. Its Salesforce landscape included Sales Cloud, Experience Cloud for loyalty programs and B2C Commerce for direct-to-consumer storefronts. The team watched uptime and error rates of its systems on monitoring dashboards built on the ELK stack, but Salesforce logs and exceptions stayed inside Salesforce.

The company also produced electronic devices at manufacturing plants. Each device came off the line with a serial number and other data that the business needed in Salesforce as assets. This factory data arrived as a daily CSV file, and someone uploaded it into Salesforce by hand every day.

What I did

I joined the internal DevOps team in Kraków as a contractor. My first task was an audit of the Salesforce CI/CD pipeline that a global systems integrator had built for the company. I reviewed it, confirmed it was sound, and moved on to other work in the team.

Next I connected Salesforce to the company’s monitoring. Using the standard Salesforce input plugin for Logstash, I set up a feed of Salesforce debug logs and exceptions into Elasticsearch, so they appeared in the same Kibana dashboards the team already used for uptime and error rates across its other systems. For the first time, Salesforce errors were visible next to everything else the team monitored, and an error could be traced end to end across systems: when something failed between Salesforce and another platform, the team could always find where it broke.

The other piece I owned end to end was the device data feed. I built a serverless pipeline in Python on AWS Lambda: when the daily file landed in S3, the function parsed it, loaded the records into Salesforce through the Bulk API and wrote the results, successes and errors, back to S3. The load ran unattended, and every run left an auditable record.

I also represented the Salesforce architecture side in a workshop with Salesforce at the company’s HQ on Experience Cloud scalability. The problem was concurrent logins under peak load, for example when a loyalty promotion launched and login traffic spiked. I challenged the proposed approaches against Experience Cloud’s login and concurrency limits and validated the target architecture. The outcome was a recommended target architecture that took peak traffic off Experience Cloud logins while keeping Salesforce as the system of record.

Alongside that I worked on the team’s platform requests, such as SSO setup, and got hands-on with its AWS stack.

The result

End-to-end traceable error logs: when something failed between Salesforce and other systems, the team could always find where it broke
Daily device data load into Salesforce fully automated, running unattended
No more manual uploads: the daily file reaches Salesforce without anyone touching it, with an audit trail for every run

System landscape

Layer	Components
Sources	Daily CSV export of factory device data (serial numbers, device attributes); Salesforce debug logs and exceptions
Integration	AWS Lambda in Python with Salesforce Bulk API; Logstash with the Salesforce input plugin
Storage & Search	AWS S3 (inbound files, load results); Elasticsearch
Salesforce	Sales Cloud, Experience Cloud (loyalty), B2C Commerce (direct-to-consumer storefronts)
Monitoring	Kibana dashboards (uptime, error rates)
Platform	AWS EC2, Docker, Rancher, Terraform, Route 53, Elastic Load Balancing

Team & duration

6 months, 06/2017 to 11/2017. Internal client DevOps team of 6 developers; I was an embedded contractor.