Dead Letter Functions

Dead Letter functions are used to correct events that have been sent to the Dead Letter due to schema issues. These functions can correct the schema issues, allowing the events to be replayed back into the system.

How Dead Letter Functions Work

When an event is sent to the Dead Letter, it is because the event failed to meet the schema requirements for the collection it was intended for. Dead Letter functions can be used to correct the schema issues with the event, allowing it to be replayed back into the system.

Dead Letter functions are written in various languages, including JavaScript, TypeScript, Rust, and Golang. These functions are executed using WebAssembly (Wasm), ensuring maximum speed and efficiency.

Creating a Collection Function

To create a Dead Letter function, you will need to write the function in the language of your choice and compile it to WebAssembly. Once you have compiled the function, you can upload it to Streamdal and configure it to run when events are sent to the Dead Letter.

  1. Navigate to the Streamdal console and select ‘Functions’ from the side navigation.
  2. Click on the + button to create a new function.
  3. Choose ‘Deadletter Function’ as the type.
  4. Write your function code in the provided editor.
  5. In the test section, paste in a sample payload. If you already have messages in the Deadletter, click on an event and select ‘Copy Payload’. Paste in everything from the payload section.
  6. Build
  7. Run test and confirm no errors
  8. Deploy
  9. Navigate back to the collection -> deadletter
  10. Create a stage by selecting all Events or specific events
  11. Select the function you just created and apply it to the stage
  12. Once the function is ran you should see all the events in the stage as ‘Replayable’

Once all the events are corrected in the deadletter stage you can replay into a collection or replay to a destination for reprocessing.

Conclusion

Dead Letter functions are a powerful tool for correcting schema issues with events that have been sent to the Dead Letter. By leveraging these functions, you can ensure that your data is processed correctly and efficiently, even when schema issues arise.