Gradient Boosting Implementation from Scratch
Project links
Skills
About this project
Inspired by the sustained popularity of the Gradient Boosting Regression Tree (“GBTR”) algorithm, we go back to square one in an attempt to complete a from-scratch implementation in Python. This paper documents a journey from the basics of the regression tree algorithm through gradient boosting and concludes with an example implementation and a controlled simulation study. The outcome is a reliable open-source code base, which in the future will serve as an excellent platform for additional improvements and optimization.