[Noob] How should I be structuring my descriptors/sets?

Apologies ahead of time if this is way too long. I’m learning Vulkan through vkguide currently, and I stopped at around the start of chapter 4 to try and figure out how to do image samplers myself. So far I’ve been able to get by on reading documentation and examples alongside debugging with validation layer error messages. But currently I’m stuck on how to handle managing descriptors/rendering multiple objects. Right now I’m drawing a hardcoded rectangular mesh and an imported GLTF model to the screen, with both using the same vert/frag shaders. Each “frame” gets 2 descriptor sets.

Set 0 is a uniform buffer that stores a mat4 that I use to translate the output vertex

Set 1 is the texture sampler.

Set 1 gets updated once before the rendering begins, but set 0 is where I’m kind of stuck. My goal was to update set 0 after drawing the rectangle so that the model is the only thing shifting left to right.

So during the render function I tried doing

vkCmdBindPipeline();
glm::mat4 shift;
VkDescriptorBufferinfo bufferInfo{};
VkWriteDescriptorSet uboWrite = {};
uboWrite.dstSet = currentFrameObject.PipelineDescriptors[0];
vkUpdateDescriptorSets(device, 1, &uboWrite, 0, nullptr);

vkCmdBindDescriptorSets(currentFrameObject.PipelineDescriptors);

vkCmdDrawIndexed(); // Draw the rectangle

// update the mat4 shift to some other value
vkUpdateDescriptorSets(device, 1, &uboWrite, 0, nullptr);

vkCmdDrawIndexed(); // Draw the imported model

Validation layers then reported that this invalidates the command buffer since the descriptors weren’t created with the ability to UPDATE_AFTER_BIND. I assume this is a race condition problem caused by modifying values that the pipeline hasn’t fully processed yet. So I’m kind of stuck on how I should be organizing these descriptors in order to render multiple different objects possibly with different textures. Should I be giving each rendered object its own descriptor sets instead and bind them between draw calls? Or something like making one big descriptor that stores all this data for all drawn objects and update it once before drawing?

Update: So at first I made two separate descriptor sets for the rectangle and the box, thinking that I could update the set separately between draw calls, basically

vkCmdBindDescriptorSets(); // the set used by the rectangle
vkCmdDrawIndexed(); // Draw the rectangle
vkCmdUpdateDescriptorSets(); // update the set used by the model
vkCmdBindDescriptorSets(); // bind that set
vkCmdDrawIndexed(); // draw the model

which also didn’t work. Then I looked again and realized that both descriptor sets were reading from the same buffer object, so I made a second placeholder buffer to hold the model’s mat4 data separate from the rectangle. This worked so now I’m able to render two objects with independent descriptor data. So I guess the idea would be to give each instance of a rendered object its own descriptor set(s) and buffer (Or some index into an array of buffers the main program keeps track of).