Q: Can I use the same model for both the main task and evaluation? A: Usually, yes. Test whether the judge agrees with human labels on a held-out test set. https://hamel.dev/blog/posts/evals-faq/can-i-use-the-same-model-for-both-the-main-task-and-evaluation.html
Q: Can I use the same model for both the main task and evaluation?
AISummary
Q: Can I use the same model for both the main task and evaluation? A: Usually, yes. Test whether the judge agrees with human labels on a held-out test set. https://hamel.dev/blog/posts/evals-faq/can-i-use-the-same-model-for-both-the-main-task-and-evaluation.html
Source: Hamel Husain · x.com