FirebaseRecycleAdapter adding onSwipe the right way

I have a FirebaseRecyclerAdapter where I try to add an onSwipe method that I created with the ItemTouchHelper.SimpleCallback. Below is my code where is try to delete a firebase entry upon swipe. The code below does indeed delete a firebase entry, except it always deletes the last entry in the list. I expect it has something to do with parsing the list_id the wrong way. Is it perhaps possible to call onSwipe inside the populateViewHolder?

private String other_id;


protected void onStart() {
    super.onStart();

final FirebaseRecyclerAdapter<Lists, ListAdapter.ListsViewHolder> listsRecycleViewAdapter = new FirebaseRecyclerAdapter<Lists, ListAdapter.ListsViewHolder>(
            Lists.class,
            R.layout.list_single_layout,
            ListAdapter.ListsViewHolder.class,
            mDoneListRef


    ) {
        @Override
        protected void populateViewHolder(final ListAdapter.ListsViewHolder viewHolder, Lists model, final int position) {

            final String list_id = getRef(viewHolder.getAdapterPosition()).getKey();

            other_id = list_id;

            mDoneListRef.child(list_id).addValueEventListener(new ValueEventListener() {
                @Override
                public void onDataChange(DataSnapshot dataSnapshot) {
                    if (dataSnapshot.exists()) {

                    //do stuff

                    }
                }

                @Override
                public void onCancelled(DatabaseError databaseError) {

                }
            });
        }
    };

    mListView.setAdapter(listsRecycleViewAdapter);

}

@Override
public void onSwiped(RecyclerView.ViewHolder viewHolder, int direction, int position) {


    mDoneListRef.child(other_id).removeValue();
}