GPTs also built by others will have their own memory. Builders will have the option to enable memory for their GPTs. 

Updated: The story has been updated with information that OpenAI is possibly building a search engine.