Umbraco 17 CMS, virtual DOM hides clicks in Playwrigth/Codex/Claude

I want to automate some data entering in the Umbraco CMS. But the way this CMS is creating with the virtual DOM gives me issues with executing normal click actions.

I tried a lot of things including injecting custom javascript into the page.

Do any of you have any tips on getting this to work, especially inside Codex Browser.

If it’s purely for data entering, I would suggest to use the mcp:

Tried that but it malfunctioned a lot, so I am trying to avoid it.

Also in this case my use case is assisted editting in a production DB so do not want to rely solely on AI unchecked visually by an editor.

Also seems textbox has issues, looks like Umbraco CMS is very AI unfriendly unfortunatly.

Hi @onlinegroeicoach

What is your use-case here? Are you trying to input some test data for playwright to use from the front-end for testing purposes? If so, what we did was create an authenticated API that playwright can call at startup which create nodes and adds data that we expect to be present when the playwright tests run. Would something like that work for you?

Justin

That is quite a statement. Perhaps you could explain what is not working or what could be improved. Or you can also add an issue on Umbraco.Ai (Issues · umbraco/Umbraco.AI · GitHub) so we can improve the AI part of Umbraco. I dare to say that Umbraco is one of the more friendly AI CMS-es.

Having said that, are you on Umbraco 17.4 or later? That introduces some functionality that should greatly improve AI and editing fields. Also, a Editor MCP is coming (instead of the developer MCP) that is more restricted and not just meant for development.

Hi @Luuk

I think @onlinegroeicoach is trying to use Playwright to amend content in the back-office and is having issues due to the shadow DOM - that is my understanding anyway?..

Justin

The issue and the goal is not yet entirely clear to me. Playwright is usually used for testing. If the goal is to have some sort of static content that needs to be the same as always, you could always use the management API to set it.

I’m also not sure why playwrite would have issues with the shadow DOM. It’s just an inline browser that can execute javascript. In the end it’s web components, so why would a shadow DOM matter for playwright?

So we really need more information about what the issue is and what the goal is in my opinion.

Hi thank you all for joining the discussion, about the MCP server tried it 2 or 3 months ago so some of the issues I had might already be solved. But anyway, my use case here is AI assisted CMS editting in backend.

Had a short holiday without laptop so sorry for the late reply, was trying out different things today and solved it. And just a sidenote, Playwright sells itself as a testing tooling but is just as well a data entry tooling.

I preferably wanted to use Codex with the builtin in browser. But I now installed the chatgpt plugin inside a regular Chrome browser instead. This can then also be controlled with Codex. And with a combination of some tricks (also for instance clicking on a field and using tab key, or pasting text in a Textarea let it type a space and then delete the space (only then it also registered in shadow DOM), I now get everything done and it is working smoothly). Most of the time now only thing the human now has to do is manually verify and press Save and Publish and do the visual check.

On my machine Playwright is very capable of just clicking around in the backoffice, absolutely no problem, no tricks needed.
But it’s Claude Code running it, not sure why Codex would be different though.

However, I must agree with other people: I don’t really see exactly the point of what you’re trying to do. I wouldn’t expect a human to sit there and wait for minutes for Playwright to fill things in on a browser.. it’s simply too boring/frustrating IMHO. What is the actual purpose of what you’re trying to do?

The MCP server is evolving as fast as possible and we’re super interested to hear about things that don’t work for you. It’s a much better option than relying on something clicking around in the browser for you.

Please report issues if you find them: help us help you :slight_smile:

I’ve run into this exact problem.

This is not really a problem with the Virtual DOM, but with how an agent might choose to interact with Playwright.

Sometimes the agent will chose to write arbitrary JavaScript and execute it inside the browser instead of using the Playwright APIs to control the browser. This will often trip the agent up, because things like shadow DOM adds complexity and means some of the JS it’ll try simply won’t work.

Add some context to explain how the backoffice works and guidance on how to interact with it - specifically to use the playwright APIs over executing JavaScript - and this problem will go away.

Firstly I look after the MCP and would be really interested to hear more about the issues that you have had. By fixing these types of issues we can make Umbraco better for everyone.

Secondly if you would still like to use Playwright I have created this Skill (which Codex will be able to load) as part of the Backoffice skills
**

I have had these Virtual DOM issues when testing extensions using Playwright and I have found this to make the interactions with Umbraco better.**
**